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What Is 
LORRI 


How reasonable is our insistence on optimal 
solutions? Not too long ago we were content with 
designing systems which merely met given speci- 
fications. It was primarily Wiener’s work on optimal 
filtering and prediction that changed profoundly 
this attitude toward the design of systems and their 
components. Today we tend, perhaps, to make a 
fetish of optimality. If a system is not “best”? in 
one sense or another, we do not feel satisfied. Indeed, 
we are apt to place too much confidence in a system 
that is, in effect, optimal by definition. 

To find an optimal system we first choose a 
criterion of performance. Then we specify a class 
of acceptable systems in terms of various constraints 
on the design, cost, etc. Finally, we determine a 
system within the specified class which is “‘best’’ 
in terms of the criterion adopted. Is this procedure 
more rational than the relatively unsophisticated 
approach of the pre-Wiener era? 

I am not sure it is. It seems to me that we have 
merely traded one kind of arbitrariness for another. 
To begin with, when we choose a criterion of per- 
formance, we generally disregard a number of 
important factors. Moreover, we oversimplify the 
problem by employing a scalar loss function. 
Vector-valued loss functions of the type considered 
by Lionel Weiss may be more appropriate in some 
cases, but their use presents many technical diffi- 
culties. In a few instances, such as the final-value 
problem considered by Booton, one can show that 
an optimal solution in terms of some simple criterion 
such as the mean-square error, also is optimal for 
a broad class of criteria. Although this is comfort- 
ing, it does not resolve the arbitrariness inherent 
in the choice of a criterion of performance. 

Another and perhaps more serious difficulty 
concerns the rational choice of decision functions 
under uncertainty. What should be done when the 
probabilities of the “states of nature’ characterizing 
a problem are not known? Here we have several 
basic principles to choose from, each having its 
partisans and opponents. The oldest principle, ‘the 
principle of insufficient reason,” attributed variously 
to Laplace or Bernoulli, states that if the a priorz 
probabilities of states are not known, they should 
be assumed to be equal. A serious and long-recog- 
nized weakness of this rule is that the states can be 
defined in many different ways, and a uniform dis- 
tribution for some particular identification of 
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states might be nonuniform for other identifications. 
Nevertheless, this principle has its proponents, and, 
in fact, in a recently suggested axiomatic system due 
to Herman Chernoff it appears as a theorem. 

If one could assume that designing a system 
subject to random inputs is a two-person zero-sum 
game against nature, then the minimax principle 
of Wald would be rational. Unfortunately, nature 
can hardly be regarded as an opponent whose loss 
is the designer’s gain. In a modification of the 
minimax principle which was suggested by Savage, 
the “payoff” to nature is the “regret,” that is, the 
difference between the maximum payoff which 
could be obtained if the state of nature were known, 
and the risk incurred by selecting a particular design 
in the absence of such knowledge. This principle, 
too, has its drawbacks, and, lke the minimax 
principle, is excessively pessimistic. 

The _optimism-pessimism index criterion of 
Hurwitz attempts to resolve this difficulty by 
taking a position somewhere between extreme 
pessimism and extreme optimism. Unfortunately, 
under this rule it is not true that a randomization 
over optimal designs remains optimal under the 
same rule—contrary to what we expect of any 
rational criterion. 

Still another possibility is to consider a set of 
criteria and select a design which is optimal under 
a majority of these. The chief objection to this 
approach is that the ordering of designs using such 
a procedure may be intransitive. For example, 
based on the minimax criterion, a design D, might 
be better than D, which, in turn, is better than D3. 
On the other hand, under Laplace’s and Savage’s 
criteria the ordering might be D, better than D; 
better than D,, and D, better than D, better than 
D,, respectively. Thus, a majority of the criteria 
rank D, over D,, D, over D,, and D, over D3, which 
is obviously inconsistent. 

At present no completely satisfactory rule for 
selecting decision functions is available, and it is 
not very likely that one will be found in the fore- 
seeable future. Perhaps all that we can reasonably 
expect is a rule which, in a somewhat equivocal 
manner, would delimit a set of ‘‘good” designs for 
a system. In any case, neither Wiener’s theory nor 
the more sophisticated approaches of decision theory 
have resolved the basic problem of how to find a 
“best? or even a “good” system under uncertainty. 
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A Systematic Approach to a Class of Problems in the 
Theory of Noise and Other Random Phenomena 
— Part ls Es annpiecs 


A. J. F. SIEGERTT 


Summary—The method of Part I is applied to the problem of 
finding the characteristic function for the probability distribution of 
J5 Soin xi(r) Kyi(r)xi(r) dr, where x;(r) denotes the jth component 
of a stationary n-dimensional Markoffian Gaussian process. The 
problem is reduced to the problem of solving 2n first-order linear 
differential equations with initial conditions only. For the case of 
constant K, the explicit solution is given in terms of the eigenvalues 
and the first 2n — 1 powers of a constant 2n X 2n matrix. For the 
case of a symmetric correlation matrix which commutes with K, 
the problem is reduced to the one-dimensional case treated in 
Part II. For the case K;;(t) = 6;16;,e~*, where the functional repre- 
sents the output of a receiver consisting of a lumped circuit amplifier, 
a quadratic detector, and a single-stage amplifier, the solution has 
been obtained in a form which is more explicit than that provided 
by the earlier methods. 


I. InrRoDUCTION 


N this part, the method described in Part I’ is 

applied to the problem of finding the characteristic 

function for the probability distribution of ¢ = 
So dose (Kj (rai(r)dr, with or without conditions 
on 2;(0) and 2,(¢), where the functions «;(7) are the 
components of a stationary n-dimensional Markofhian 
Gaussian process 2(7). 

The older method of attacking this reduces it to the 
problem of finding the solution of a certain integral 
equation. Special cases, for instance the probability 
distribution of the output of a radio receiver with square 
law detector and the probability distribution of the 
filtered output of a multiplier,” have been treated in this 
way. Another special case of interest is the joint distri- 
bution of the sample cross-correlation coefficients. 

In its original form the older method required finding 
the eigenvalues of a homogeneous integral equation and 
the calculation of the Fredholm determinant as an infinite 
product. It is possible to avoid the latter part of the 
calculation by solving instead an inhomogeneous integral 
equation, which is essentially the equation for the Volterra 
reciprocal kernel; for the special case of the Markoff 


* Manuscript received by the PGIT, August 16, 1957. 

+ Northwestern University, Evanston, Ill. and consultant for 
The RAND Corp., Santa Monica, Calif. 

1D. A. Darling and A. J. F. Siegert, ‘“A Systematic Approach to 
a Class of Problems in the Theory of Noise and Other Random 
Phenomena—Part I,” The RAND Corp., Santa Monica, Calif., 
Paper P-738; September 10, 1955, and IRE Trans. on INFORMATION 
THEORY, vol. 3, pp. 32-37; March, 1957. See also, A. J. F. Siegert, 
“Part I, Examples,’”’? The RAND Corp., Paper P-730; September, 
1955, and IRE Trans. on Inrormation THEorY, vol. 3, pp. 38-43; 
March, 1957. 

2D. G. Lampard, “The probability distribution for the filtered 
output of a multiplier whose inputs are correlated, stationary, 
Gaussian time-series,” IRE Trans. on INrorMATION THEORY, Vol. 
2, pp. 4-11; March, 1956. References to earlier publications using 
the older method are given in this paper. 


process, the integral equation can be reduced to a differ- 
ential equation.® However, the actual determination of 
that solution of the differential equation which also solves 
the integral equation requires a rather tedious procedure 
of satisfying boundary and matching conditions even if 
the differential equation can be solved. It is not surprising, 
therefore, that the number and type of cases which have 
been solved explicitly have been very restricted. 

While there are approximation methods valid in 
limiting cases, the exploration for cases which are exactly 
soluble seems to be of importance, since in Part I a 
perturbation formalism was derived, which permits the 
calculation of the characteristic function for functionals 
‘Gn the neighborhood” of one for which the solution can 
be obtained. For this purpose, one needs not only the 
characteristic function of F, but also a function # which 
is similar to a joint characteristic function and defined as 


f(m, Up BO Na eineee a 3 a3 t, d) 


= (exp ‘i » (ne(0) — E,2,(1)) = as} 


In Part II we studied this function for the Uhlenbeck 
process a(t) and § = fj K(r) a°(r)dr and found that it is 
of the form 


#(m, &; t,\) = f-exp {—4(an’ + 2bé + c&)}, (2) 


where f, a, b, and ¢ are functions of ¢ only, and satisfy a 
system of first-order differential equations with initial 
conditions only. The function ¢ specially was found to be 
the solution of a Riccati equation, which could be 
converted by standard procedure into a linear second- 
order differential equation for a function wu, again with 
initial conditions only. If wu could be determined, then f, a, 
and b could be expressed in terms of u, by quadratures 
or more simply; we found, for instance, that f was given 
simply by 


(1) 


Av 


{(@) = {uO0)/u)}”. 


Since f was expressed by the older method as the reciprocal 
square root of the Fredholm determinant of the kernel 
e "'™""2! K(z,), this showed that the Fredholm deter- 
minant of this particular kernel satisfies a second-order 
linear differential equation as a function of the upper 
limit of the integral equation. Furthermore, the functions 


SA. J. F. Siegert, “Passage of stationary processes through linear 
and non-linear devices,’ IRE Trans. on INForMATION THEORY, 
vol. 3, pp. 4-25; March, 1954. 


, >, and ¢ are essentially the values g(0, 0), g(0, #), and 
(, 1) of the Volterra reciprocal kernel g(r:, 72); one 
us had the result that these special values of g can be 
btained trivially from the Fredholm determinant, for 
he special form of the kernel,* so that it is not necessary 
solve the integral equation in order to obtain these 
pecial values of g. 
In the present paper these results have been generalized 
the case of quadratic functionals of a stationary, 
‘dimensional Markoffian Gaussian random process. The 
nalogs of the functions a, b, and c are now matrices of n 
pws and columns, which are functions of ¢. In Section II 
ye obtain two first-order linear differential equations, 
yith initial conditions only, for the matrices uw = b-* and 
= cb *. The Fredholm determinant of the older method 
cow reduces, but for a trivial factor, to the determinant 
f uw. In Section III an explicit solution is given for the 
ase of constant K, which requires only the finding of 
is eigenvalues and the first 2n—1 powers of a 
ertain 2n X 2n matrix with constant elements, or 
ternatively the eigenvalues and eigenvectors of this 
hatrix. The case where the correlation matrix is sym- 
hetric and commutes with K has been reduced in 
section IV to the one-dimensional problem. In Section V 
he principal equation of the present method was derived 
‘irectly by means of the older method. The characteristic 
uinction for the probability distribution of f¢ e ‘x()dt 
i been obtained in Section VI in an explicit form, as 


e reciprocal.square root of ann X n determinant whose 
lements are series of hypergeometric type. 
| The linear first-order equations, of course, are in gen- 
ral not easy to solve. They seem to be, however, a more 
<onomical formulation of the problem, freed of the 
.ecessity of solving the integral equation, in a case where 
ne is interested only in the Fredholm determinant and 
ertain special values of the reciprocal kernel. Therefore, 
hey also may provide a more convenient basis for 
‘umerical computation. 
| II. RepuctTIon oF THE PROBLEM TO LINEAR 
| Frrst-OrDER DIFFERENTIAL EQUATIONS 


The n-dimensional stationary Gaussian process X(t) = 
0, (t), v2(t), --- 2,(t)} with mean zero is described by 
\) correlation matrix R(r) with components’ 


| Ryilr) = (ax(Har(t + Te) ae paz) (3) 
| R(1) = R(-2) (4) 


yhere the tilde denotes transposed matrix. or stationary 
Aarkoffian Gaussian process, P(r) has the form 


4 Some of these relations could be shown to be independent of the 
ecial form of the kernel. Recently, R. Bellman obtained a Riccati 
pe integrodifferential equation for the Fredholm resolvent of a 

edolm integral equation; see, ‘Functional Equations in the 
heory of Dynamic Programming-VII: An Integro-Differential 
quation for the Fredholm Resolvent,’”’ The RAND Corp., Santa 
onica, Calif., Paper P-859; May 7, 1956. 

DIME Wang and G. E. Uhlenbeck, “On the theory of the 
rownian Motion II,” Rev. Mod. Phys., vol. 17, pp. 323-342; August, 
45, 
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or 


R(7) = e** (5) 


for r > 0, if R(O) is the unit matrix.® 
The characteristic function yx. of the joint probability 
distribution of X(0) and X(r) is then 


xX2(m; cee & t) 


= (exp E ME, (m.t,(0) a eand) |) 
== fe) {4 bs (ni a £1) si 2 DE ndtul dé |} (6) 


The characteristic function x. satisfies the differential 
equation 


Lito pts a aes 


OX2 
Ot 


OxXs 
ay >S, ( x i Eoxs Ju (7) 
Pall 0&; 
which can be verified directly using (5). 
To obtain the characteristic function for the distribution 
of the functional 
g= f Da)Ka(eaie) ar ®) 
0 q55 
[with or without conditions on X(0) and X(f)] where K 


is a symmetric matrix, we define the characteristic func- 
tion for the joint distribution of X(O), X(¢), and wu, 


E,, t, d) = \t pz (ma.(0) 
= £,a,(t)) a ASF} ) ay (9) 


where A is taken to be real if § > 0, and negative imaginary 
otherwise. 
Proceeding as in Parts I and II, we then obtain 


BG wie: Bessa 23K (exp 


oF 
Afihat > ( OE, fe Es \Qnt =X » IS i) eee ie ae (10) 
The initial condition is obtained from (9) as 

fr-o = exp —— ey (te -F Ey}. (11) 


Eq. (10) is reduced to a set of first-order differential 
equations with ¢t as the independent variable by the 
ansatz 


fia aes (12) 
where f is a function of ¢ alone, and 
o= 5 »y [nidin(L) ne | Qnibi(D& a EiCin( Ec (13) 


1,k 


where a and c are symmetric matrices depending only 
on t. Eq. (10) then becomes 


f4ié- D (2° + & Jud 


(14) 


ao ag é) 
{ee cae 3g, + ab, aE; 


6 This can be assumed without loss of generality; see zbzd, foot- 
note 19, p. 330. 
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where dots indicate differentiation with respect to ¢. 
Using the symmetry of the matrix c, one has 


ge, =~ bs — § Do Gieus F eb) 

= —Lilnidis + be) 05) 
and (14) becomes . 
dln. f/di. = 2. ye (nidepne ie on bake ee) (16) 


ex. (nibiz + E:¢i;) — E)Qin 


a 


a ON Dekel on se 8 Geos 
7,0 ik 


+ €,€:;)(mbia + E,Cx1) | 


—xr SS Cie a ON DE ni(b;;K 5.01%) 
il VaR we 
3F SD nil— ME b:;Qix + 2d ye b Kir lé& 
Mas 7 il 


ar 2 ee ye Ci Qin + Qin) 


=m es Cj; K j Cre | 
il 


—) tr (Ke) + XD) 0.(bK0) sve 
ae ye ni[—(bQ) ix. + 2(bKe) i lé, 


he 2 E[—(cQ) i = Qix Ae NCKO) ix JE 


—dtr (Ke) + 3D n(0Kb + OKI), 
i,k 


ag nil—(0Q) ix ae 2 OKC) ;x JE, 


i,k 


oa 


+ 4D t(-cQ — Qc + OH Q + WcKo) né, 

i,k 
where the second and fourth terms have been written in 
symmetric form in order to facilitate comparison of 
coefficients, and where tr indicates the trace of a matrix. 
One thus obtains 


aus 


-F —) tr (Ke) (17a) 
é= —-Q+Q)+cQ4+ Qc — 2rcKe  (17b) 
ad = —\(bKb + bKb) (17¢) 
b = b[Q — 2nKc]. (17d) 


The initial conditions are obtained from (11)—(13) as 
f(0) = 1 (18a) 
CO) r= O(Ohie== (0) cal (18b) 


I 


where J denotes the unit matrix. 

The scalar equation (17a) and the matrix differential 
equations (17b)—(17d) reduce to (14a)—(14d) of Part II for 
the one-dimensional case. Eqs. (17a) and (17c) can still be 
solved by quadratures, but (17d) is no longer as simple as 
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the corresponding (14d) of Part II. It turns out, however, 
that the solution of (17d) depends on the solution of the 
same set of linear equations as (17b) and therefore we will 
start with the latter. 

Eq. (17b) is a matrix analog of the scalar Riccati 
equation, but the familiar substitution leading from the 
latter to a second-order linear differential equation does 
not work for the matrix equation. A similar substitution, 
however, leads to two linear differential equations for two 
matrices uw and v. We write ¢ in the form 


(19) 


Cc = vw 


and obtain 


+ -1. -1 
vU — vu UU 


—(Q + Q) +w'Q 
+ Qu? — 2dou* Kou’ (20) 

& 
d= Q —(Q4+ Qu + ow "G+ Qu — 2K). 


Since we are free to choose one relation between wu and 2, 
we can let the factor of vw * be zero and obtain the two 
linear differential equations 


b= Q—Q+ Qu 
u = 2\Kv — Qu. 


(21) 


(22) 
(23) 


The initial condition is w(0) = v(O), since c(0) = J. The 
matrix Riccati equation (17b) is thus reduced to a pair of — 
first-order linear differential equations for the matrices u 
and v, 2.e., to a system of 2n first-order linear differential 
equations. 

The matrix b turns out to be simply 


b() = uu (0), (24) 
since (17d), (19), and (23) yield 
b = —biw' (25) 
oF 
bu + bu = 0 (26) 
and, therefore, 
bu = b(O)u(0) = w(0). (27) , 


The function f also is expressed simply in terms of the 
determinant of w 


Ss du;, 0 det u 


7 Le ous, (28) 


d 
ai det Uu 


One 
SS Tae (u~’) «x det U, 


ik 
where det w denotes the determinant of wu. We thus have 


d In det u 


ie te eS = tr (Qh en 


I 


(29) 


| 


—trQ + 2d tr Ke 


Glneiaee! al ld 
| ie —5t@ — 97, In det u. (30) 
Since f(0) = 1, we thus have 
/ — p ftrQ/2 det u(0) 


The matrix u(0) can be chosen arbitrarily, as long as 
ithe matrix (0) is chosen equal to u(0). 

One could have obtained (22)-(24) and (19) directly, 
using (17d) to obtain 


dt dt 
—(Q + Qb* + cQb* + Geb™ 

— 2dcKeb* — c(Qb* — 2\Kcb"') 
—(Q + Q)b™ + Qcb™ 


so that one has two linear equations for b-* and cb’. 


abe” =I 7 =a Zapy at 
pute Ke a) 
and (17b) to obtain 
| CCE parece ete 
rich) C0 tee (33) 


I 


III. Taz Case or Constant Marrix K 


| This case is of interest since the characteristic function 
lof & defined by (8), is for constant K, closely related to 
‘the characteristic function of the empirical variances 
and covariances of the components of X(r). 

_ The solution then can be reduced to the solution of 2n 
homogeneous linear algebraic equations, by writing 
(22) and (23) in the form 


(') = mM (") (34) 
U u 
where Mt is the 2n X 2n matrix 
es ( Q -@+ °) (35) 
2K —Q 
(Seript capitals will be used to denote the 2n X 2n 
matrices. ) 
_ Choosing the initial conditions u(0) = v(0) = J, we 
then have 
U If 
and 
“= (ens, ae Cae (37) 
ee ae a (38) 
Ca (ee Nes 


where (e™’)., and (e™’)o. are n X n submatrices of e*’, 


ybtained by dividing e™’ in the form 
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The unconditional characteristic function f(t) for the 
distribution of S(t) is obtained thus from (31) as 


1) = °F ldet Cn + Yall. Ge) 
If the matrix 9M can be diagonalized, 

WHS Cape, (40) 
where © is a diagonal matrix, the matrix e™’ can be 
evaluated by using 

C= Cre ©, (41) 


where e”’ is the diagonal matrix with elements exp (0;;t). 

Alternatively, one can evaluate e’ by the following 

procedure, which does not depend on the possibility of 

diagonalizing 91%. Let D(z) be the secular determinant of 91, 

D(z) = det (2I — 9M) (42) 

which is a polynomial of degree 2n in z. Define the function 
o(u, z) by 


Dw) = D@, 
u—z 


o(u, 2) = (43) 
[The explicit form of ¢(u, 2) is given in (49).] 
Since (u — 2) is a divisor of D(w) — D(z), the function 
o(u, 2) is a polynomial of degree (2n — 1) in both p and z, 
and is symmetric in » and z. We then have — 


pt O(L, JIT) d 
D(u) 


where the path of integration encircles all roots of D(u), 
and the integral can be evaluated by the method of resi- 
dues and appears as a polynomial of degree 2n — 1 in the 
matrix IW with time-dependent coefficients. The fact that 
e™™ has this particular form can already be concluded from 
the Cayley’ theorem, which states that 


DO) = 0 


em — 1 
Qrv 


(44) 


(45) 


and allows expression of 9°” and all higher powers of SW in 
the power series for e””’ by the first 2n — 1 powers of SM. 

That the coefficients are correctly given by (44) can 
be seen by differentiating (44). One obtains 


d Me 1 t (u a MW)o(u, SU) { 
is ; ate om é D(u) 3 du (46) 
= ai PO du = 0 


To complete the proof, one need only to show that (44) 
yields the unit matrix for ¢ = 0. We transform to a new 
variable € = 1/u and can contract the path of integration 
to a small circle around € = 0, and obtain 


** gE", SI) 


oie 
ED(E') 


Qt 


‘OF, 
(uy, MN) d 1 


D(u) ISS ue a 


7R. Courant and D. Hilbert, “Methoden der Mathematischen 
Physik,” Springer Verlag, Berlin, Germany, vol. 1, p. 18; 1931. 
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Now if We thus obtain, with 
= Aj; = —8; 57 
De) = >, bz’, (48) * f “ 
28 and 
then Kj; x;(t), (58) 
2n s ae 
ou; 2) = a the equation 
s=1 KB @ 
Qn s—1 ie d ( ul ar melee ee 0 59 
= PLipepro cs 3 OF at uc) Ne eee CF 
=1 r=0 
2n-1 Qn or with 
re r Qala 
NE ae (a) One (60) 
and the equation 
Or o(e MU) oe cs ec 3 BES oe Bik, (t(x,)) @a;,/dx; + di;,/de; — tj. = 0 (61) 
re DE) dg = $ rs ; aa , which are the same as (21) of Part II, and were solved there 
E ~~ bE explicitly for some special forms of k,(t). 
ne The initial condition u(0) = v(0) requires only U(O) = 
= $ dE Donl  ECbanM Ff Bona) F+W (0) and the initial condition for (61) is, therefore, 
E Don a EDon-1 SF ay ak Ae 
a Bre bs 
= or (2) =) 2NIS( Ova aos (62) | 
IV. REDUCTION OF A SPECIAL CASE TO THE If «;(0) does not vanish, this simplifies to 
Onr-DIMENSIONAL PROBLEM e 
Uj fc 
If Q is symmetric (which implies real eigenvalues), it | tine | pee [Wir }e=o- (63) © 
can be written in the form Rory 
re Without losing generality, one can choose 
Qr= Or="S ANS (50) 
: . ; : Ji: : [W,-o] = I; (64) 
where A is a diagonal matrix, since by definition Q is real. 
It turns out that the matrix S need not be known ex- then @ is a diagonal matrix for all «x [provided that 


plicitly. Eqs. (22) and (23) then can be reduced to 


V = AV — 2AU (51) 
COON i NEI (52) 

where 
Veo Ue Stic) Shaws ao. (53) 


Especially if K commutes with Q, the matrix S can be 
chosen so that K is diagonal and, if its elements do not 
vanish identically, (51) and (52) reduce to those of the 
one-dimensional case. If «@ is defined by 
(54) 


= A 
ea oe (Ue 


then 


a= e(U + AV) = c'QKV. 


(55) 
Since K was assumed to be diagonal, we then have 


1 
2 


1 -1 d z\—1 = Aji At 
i A ai Sa 1 [ann t| = e A rf + i}e V) 


=e. 1h. (56) 


8 If some elements of K vanish, the corresponding rows of U are 
obtained directly from (52) and the initial condition as U;, = 
eAjjt 6;x, otherwise the calculation is unchanged. 


x;(0) < o] and one has 


n -1/2 
f() = iT w,et0) 


with a; ;(x) defined by (61), (63), and (64). 


V. DERIVATION OF THE Matrix Riccart 
EQUATION BY THE OLDER Meruop 


Since #;;(r) is the autocorrelation function of 2;(t), 
it can be written in the form 
R;i(7) = if h,Ohj(r + B) dd; (65) 


where one may demand h; (i) 


= 0 ford < 0. One then 
can write x;(t) in the form 


ni) = Doe f hilt = By.) ao, (66) 
where the coefficients c, are independent Gaussian random 
variables with mean zero and variance unity, and where 
the functions y,(%) are a complete orthonormal set. 
Then the correlation matrix is given by 


Raa, = ie h(8)hj(t + 8) de. (67) 


ow let the functions y,(t) be the eigenfunctions of the 
ntegral equation 


lusing (66), (68), and (69). 
_.One then obtains for the joint characteristic function 


(exp E Dia Oee) 
| ; 4 
= i | 2 K,,(1)2;(7)2,(7) in|) 


(71) 
= (exp E Sov, —rA Dd or) 
= [1 (+ 2)” -exp |-3 ye = 20.) | 
svith the abbreviations 
VW, = DY [n9;,(0) + 61,1 (72) 
and 
gilt) = [hilt — d)¥,(0) do. (73) 


The function f and the matrix c introduced by (12) and 
(13), therefore, are given by 


fs (Casha (74) 

and 
| dry(b) birt) ia 

oe =e i) kay 78) 

We now define the matrix function g(7;, T2) by 

dur(T1)P1,( 72) 
— Geilt1, T2) = S 
. 7 1 + 2)d, (76) 
| = I WAms — H)hilre as O)V,, 0.) dd,dds, 
where 
| 
| Vv, ()W (Os) 


Using (68), one sees that 


1(,9) +2 [ Nr, 8000, 9.) dd = a, — 8.) (78) 


vl) = [MW 8) ¥(6.) dds, (68) 
where 
(9, 3,) = i | D hi — WK) — 9) dd; (6D 
|_ 

2, if : «(OK ;(0)2x,(8) dd = Deeds, (70) 
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and, therefore, 


gii(ts, T2) + 2d Il Ge 


hit.’ — 3s) dd dd, dd» 


3B) AG, 3)yO, Ie) 


= xe — b1)hi(t2 — 04) dd; 


= gilt, T2) 


-+- 2X / Ga 0) is Dy hj(r — 01) K j(7) oe 


SG HY, Bo) hi(re — d,) dr dd dd, dd, 
Gii(T15 T2) 
+r fo ORs — Kaur, 1) 
0 ik 


ae Ril re aa 7); 


using (76), (69), and (65). 
In matrix form this equation is written as 


g(r, 7a) +2 [Rr — 1) KC) gle, 73)dr = Rr — 1) (80) 


and g(t, ¢) is the matrix c(t). 
The iteration solution, written for 7, = 7. = t, becomes 


Ge ST, (-2r* | HU, )RG= pdr en 
1 0 
where 
H(i, 7) = HG, 7) = Re = a) V2) 
and . 
Ter / Ho, DG a) dee 
Differentiation yields 
ee ee =D 2h) {HG t) 
: WEES T) 
+f | en Re-7) (4 
+ A(t, DRG — 0 | ar} 
where 
dH (Ga) Cans A 
dt = mal a f AAG, 7) H(11, T2) 
oe (Teas a) OT. 9: OTean 
bs, (85) 
=o HS Gt aGe7) 
+. Ha) a Eee aCe t) at. 


0 
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Using (82), (4), and (5), one has 


LD = OH, 2. (86) 
Substituting in (84) and using (81), one obtains 
20D = Qeglt, ) — D + (lt, 9 = NO 


Sos (2H i 


= SHE, t) i) Hes? (Goes) ar}. (87) 
k=1 0 
It is convenient to define 
[ HG, 20 ar = 60 (88) 
0 


for any function ¢(7); then the last sum can be written as 
» (—2h)" > H(t, 0 (E H(t, )R(t — 1) dt 
= = Ds (—2a)""'H(, 0) ik HOG, )R(t — 7) dt 
= > (2h) ee teed) 
Sy (—2)! fh FOU DRE ar 
= -2.| Ko + a (—2r)"H "VE, 0 | 
1c: + (=) f H(t, OR — 7) ar| (39) 


using (82) and (88). We recognize the second bracket as 
g(t, t) and the first bracket as g(t, t) K(t), since 


R(t — jNK() = HU, b (90) 
and therefore 
Gt) = Io (4) 
+ 2 (—2n)” rE H(t, yH(1, t) dr. (91) 


Now writing c(t) for g(t, ) we obtain from (87) and (89) 


GAO 0c) OO Le 


dt (92) 


in agreement with (17b). 


VI. CHARACTERISTIC FUNCTIONS FOR THE PROBABILITY 
DISTRIBUTION oF J e' x? (é) dt 
For the calculations of this section it is useful to have 
(22), (23), and (81) in a different form. 
Since 


eos 


= dete °%, (93) 


March 
(31) can be written as 
f7() = det [e°'u(#)] (94) 
when u(0) = v(0) is chosen to be the unit matrix. 
From (23) it follows that 
© [eMu(t)] = Qe" K (O00 (95) 
and 
ou) = ten | e' K(t)u(t’) dt’. (96) 
0 


This equation is specially useful in the limit t > o, 
since the integral, in this limit, can be expressed in terms 
of the Laplace transform of K(¢’) v(t’) with — Q sub- 
stituted for the Laplace variable. 


With 
Vit) = [Kon (97) 
and 
Vii(s) = te Go Vai) dt, (98) ; 
one has 
lim [e? u(t) Jer = On] + 2r > 6 \5 Veen) dt 
= On + 2» sy i [o> Waves dt 
= 64 + 2X D [Vins(—Q) len. (99) . 


The derivation shows that the last expression is to be 
evaluated as follows. The matrix elements V,,(s), which 
are ordinary functions of s, become matrices V;,(— Q) 
when — Q is substituted for s. These matrices have 
elements [Vx:(— Q)]x., and the summation is to be carried 
out over the index h, to obtain the (kl) element of the 
matrix lim,... (e°’ u(t) — I). If the matrix Q is available 
in diagonalized form 


Qn a 2 Soe (100) 
af 
the above result can be written as 
lim [e’u(é)]ar = S42 + 2A D7 SiS" VBy))y: C01) 


tro Y 


which can also be derived directly from (99). Another | 
form of this result, which is useful in the special case 
treated below, is obtained from (44), with Q substituted 
for Iv. The integral in (99) then becomes 


fevou=f ger vy a 


D(u) 
1 (u, a 
7 on ne V(~») dp (99a) 


where the path of integration encircles the roots of 


958 


DW) = Te + 80) (99b) 


iP where (— £,) are the n eigenvalues of Q. If these are 
ull distinct, one has 


i e*' V(t) dt = Lel-6,, Q) V(B,) (C185) O90) 


The matrix v(t) satisfies the integral equation 


Q(t—-t’) 


(t) =o 8 4 On iP & 


— 8! !1KNo(#) di’. (102) 


To check this, one multiplies from the left by e 9% and 
Hifferentiates and obtains 


le Md] = So He 


+ 9 “ aes in ec? K(t/)u(t’) aw} 
0 


= an o ‘ eo 8 K(t/\u(t’) dt? 
-£ ee (1 = rn | e °""*K(t)o(t’) aw’), (103) 


since the terms arising from differentiation of the integrals 
rancel. Using (23), one then has 


| £ eM (0) = Ste 


= 6 *Q + Qult) 


O19 9°) 62 'u(4) 


(104) 


in agreement with (22). 
Instead of considering (102) as an integral equation 
For v(t), one may consider its Laplace transform 


(8) = (I + Q)” 
+ 2n{(sI + Q)* — GI -— QV) 


AS an algebraic relation between 6(s) and V(s) to be used 
for the determination of é(s) and V(s), together with 


(105) 


Vie = if * e-"K (v(t) dt, 


which follows from (97) and (98). 


As an example of a case in which this approach is useful, 
| A . 
we now consider the special case 


(106) 


| Ki@) rashes (107a) 
where R is the constant matrix with elements 
| Ria = 6n61- (107b) 


The case K(é) = Re *’ of course, can be reduced to this 
case by choosing a as unit of time. Eq. (106) then 
reduces to 


Vis) = Ri(s + 1) (108) 
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so that only the elements of the first row of #(s) are 
needed for the evaluation of (99). For these, (105) reduces 
to the difference equation 


6,,(s) = a,(s) + 2db(s)b,,(s + 1) (109) 

where 
a(s) = (I + Q)it (110) 
b() = {Cl + Q) — Gl — Q) Su. (111) 


(The tilde indicating the transposed matrix can be 
omitted for the diagonal element.) 
The functions a,(s) and b(s) are rational functions of s. 
Although there are more elegant methods of solving 
(109), the most transparent result is obtained by iteration, 
which yields 


iu) = a) + DEY [Lot m-ale+) (112) 
and 
bust) =ae+1 

+ Ly’ [ve + mae+i+). (18) 
This can be written as 
ius) = Ly TL o6 + mals +147), (1188) 


if we adopt the convention that products containing no 
factors, such as [[°_,, are to be replaced by unity, in 
this and all subsequent equations. 

The terms of the series can be obtained in a more 
explicit form by the procedure used in Section III, (42) 
et seg. Let D(a) be the secular determinant of Q 

D@) =det@I-Q =[]@+s8) 14 
K=1 
when Q is a matrix of m rows and columns with eigenvalues 
(— £,). The eigenvalues are assumed to be distinct. 
Let v(x, y) be the polynomial of degree (n — 1) in both 
x and y defined by 


(c — yo, y) = D@) — Dy). (115) 


The matrices used in (110) and (111) then can be written 
in the form of polynomials in s and Q 


(sI — Q)* = ¢(s, Q)/D(s) 
(I + Q) =—(-sl — Q)” =—¢(—s, Q)/D(-9). 
The function b(s) is thus 

bs) = —[e(—s, QuD® + ¢(s, Q)D(—s) 


(116) 
(117) 


//DQ@D(-9). 
(118) 


Since the terms with the highest exponent of x in 
v(x, y) and D(a) are w”* and 2”, respectively, the co- 
efficient of s°*”* in the numerator vanishes and the 
numerator is a polynomial of degree 2(n — 1). It could be 


12 


of lower order only if Q;,; vanished, which is a case of no 
practical interest. Only even powers of s can occur, 
because the numerator is obviously an even function of s. 

Let the roots of the numerator in (118) be + a, 
(ere ahs n — 1). Then the function b(s) can be 
written in the form 


5) 


n—-1 


const - [| (s’? —‘a;) 
=1 


B= 


bis 


/Tle-8. a9 


The constant is easily determined by expanding (111) 
for large s. One has 


b(s) = —2s*Qn ate (120) 
and, therefore, 
Gail n 
b(s) = —2011 ita (s° =F Oy) (s° ae Bor (121) 
p=1 Kk=1 


In order to obtain the product in (112) explicitly, note 
that 


tetera =T(strt+1-—a)/I(s+t1l—a) (122) 


m= 


and, therefore, 


1 b(s + m) = (—2Q11)" 
gnosis 1 —a,)I(s +r+1+a,) 
=a is + 1-—a,)r(s +14 a,) 


TI MOSS poallsea/ eM Coie al Se) 
RCS rae P26 Wee ed ers, 
where again for n = 1, the first product is to be replaced 
by unity. We need this expression only for s = 8,, and can 
take out of the second product the term « = y 


Erg renee, 
rl Td +r + 28,) 


(123) 


K=1 


I] 6, + m) = 


IR Gye Pest roy) IG eee ae 
w=1 (Gs a= a) (By = ib = Qty) 
= IM oigdet ete CONC sh Am eie) 
: \ al 
Ute+i+e,—e)re+i+s,+a) “4 
The functions a,(6, + 1 + 7r) can be written as 
Gis et 
a g( By 1 r; Q)1:/D(—-B,y lr r) (125) 


(—)""e(—B, — L— 7, Das I]@+1+ 8, — 8.) 


by use of (110), (114), and (117). The denominator of this 
expression combines with the factor 


PUG CSP alee y ere 8) 
oe 
in the preceding equation. 
Eq. (99c) becomes, in the special case defined by 
(107a)—(108), 


> 
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co 


[ "VOI at 


n 


rH bs. A—Bas Q) ei Gy = 1) 


y=1 
KAY 


Combining (113a) and (124)-(126), one obtains 


lim [e° WO) Jar = 6x. + 2d De, is, QD) 


| 


March | 


/ Il @. - 8). (126) 


J 


| 


#.(—40Q1:)" Tr + 28.) \ oe 
De Ge Ce eesae an) o(—8, —1—1, Qi 
pp Retest 1s aT, +r 1+ ay) 
mat (Gy + 1 = a) By Lea) 
oe IG, = 8)Fd +p 2B) | | 
lligre+s, mle +1 ele a 


where ¢ is defined by (115), and the numbers + a, are 
the roots of the numerator in (118), and (— 6,) are the 


eigenvalues of Q. A product without factors, such as | 


[[%3} with n = 1, has to be replaced by unity. 


For n = 1, this reduces to the result obtained in (54) of — 


Part II. In that case, the matrices reduce to numbers, and 


Q=QQ1n = —-f (128) 
D(x) = (x + Bi) (129) 


LY 
and, since the products contain no factors, the result 
reduces in the one-dimensional case to 


f*(e) = lim [e*'u(t)] 
& = (4dB,)" CU =e 26;) 
spas Zoe OTd Er Poe 


r!T(r + 28,) 
T'(28,)(—4) 8) Tog nV ONG lea 


With 6, = 6/a this agrees with the result quoted above. 
If the diagonalizing matrix S defined by (100) is 

available, a matrix slightly simpler than that given in 

(127) can be obtained, which has the same determinant. 
Let the matrix F be defined by 


[One a ye Si: lim fe? u(t) nr Si, 
kl 5 


to 
and note that the determinant is unchanged by this 
transformation. The same procedure applied to the matrix 
elements on the right-hand side of (127) results in 
yy, Su.g(—By, Q) ia = e—B,, ne Ba) Sor 


k 


r=0 


(133) 
and 


D A Cal 6 rent ec) eS 
a S1.¢(— By ay Ui Oye 


(132) | 


(134) 


958 


From (114) and (115) one obtains 


o(@, — 6.) = D@)/@+ 86) = []@+s8), (185) 
p(—By, — B) = TG. — By) = bye TG — 8.) (136) 
und 
-8,-1-7r,—-6)=T]@-—8,-1—n. (187) 
je of these results in (132) yields 

lee = Oar == 2NGeaS sibae (138) 
with 
Her a do (—40011)" 
Gre (Green a Rae ayer: 1) 
rea Derwent UT Bes ay 1) 
TI (Cito el Gat Ore U) 
Bore teeta) Beet Bat rite 1) 
= (ie ae Pee et a at (139) 


The series is of hypergeometric type.” 

In the Appendix (det Ff)” has been computed. to 
second order in \ and the result checked by direct compu- 
tation of the first and second moment. 


VII. APPENDIX 


| To check the result given by (188) and (139), we 
compute f() to second order in \ from these equations 
and directly. 


Writing the matrix F in the form 
F=I1+)G4+NH+- (140) 
with 
Ge 285 Sube = Bel) (141) 
H,, = 283:8:(—4Q1) T] (@, + 1° — ail 
TAGS teal) eae6, Ba (Bact e2) «1 42) 
fe have, to second order in X, 
f(o) = det Fy” = exp {—3 tr In F} 
| = exp {—4tr AG + N(H — 4@)]} 
Lye 
—e i 50 G 
Ltr G@ — X(trG@)’*] + (143) 


| 
2 
— * tw H - 


9G. N. Watson, ‘‘A Treatise on the Theory of Bessel Functions,” 
Cambridge University Press, New York, N. Y., p. 100; 1944. 
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where tr stands for trace. The individual terms are evalu- 
ated as follows 


inG = 5) 2848) = 2 (144) 
tr Ga = Sy GaGa 
== Al ye Sais Bisel AF De yea Cy 
‘(1 + 6, — B,)* 
=-2> hiceinl i + : | 
oT 1+ 8, — 8B, 1+ ,.,— 86, 


= 20>) SaSi(l + 8 + Q)ii 
+ S) 8ui8.. + B + Qa) 


=u Y) Sh Sea 


= 4faU — Qu (145) 
where a,(s) is defined by (110). 
tr H = —4Q11 ss Sot 
TDG. + D* — ag) TT, + D* — 62] 
= 2 D7 SuS:.b(8. + 1) 
=F 2(bU a Q)]u (146) 


where the explicit form of b(s), given in (121), has been 
used. Substitution of these results in (148) yields 


fo) =1-r»—% {260 — Oh 


= 2lat = Nhe — 1) Ae 


= -av+3 41 ee a) 
see! car) lens tas (147) 
The first two moments of 
s= aan (148) 


where 2, (t) is the first component of the stationary 
Gaussian process specified by (3)—(5) are easily obtained 
by direct computation. 

The first moment is 


(Spy i = / OB CHEN Olea lh (149) 
0 
The second moment is 
Sa) ie i e* ei(t)ai(t’) dt i) 
; Av 
co t 
aD | - fl oar(t)a%()) x. dl’ dt. (150) 
0 0 
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For a Gaussian random function, one has 
(av(t1)a(te)ar(ts)a(ts))av = (e(t)ar(te)) avhar(ts)@(ta) aw 
=a CCEA) RCAC LAU) re 


Te (t,)ar(ta)) av(w( te) (ts) dav (151) 
and, with % = % = ¢ and t,.= ta —st,and (at 
(a2(t)22()) av = ACU) avai) aw + Well) ax(O) ov 

a Oi eet ce (152) 
Substitution in (150) yields 
Gi Pe] a] ee New 
0 0 
aoe 2 hes £(69),.} dt (153) 


March. 


by virtue of the folding theorem for Laplace transforms.” 
Using (100), one obtains. 


(Fae = 142 D1 SSS SCL + Be 8) 


or, with a,(s) and b(s) defined by (110) and (111) 
(Fa = 1424670 — Q) = 0 Om. 


We, therefore, have to second order in \ 


(154). 
(155) 


foo) = a, = 1 = NB)aw $F Fae + 


=1-d45 {1 + lal — @ 


i bU ea Q) is} ela 
in agreement with (147). 


The Axis-Crossing Intervals of Random Functions—IV 


J. A. McFADDENT 


Summary—This paper considers the intervals between axis 
crossings of a random function £(¢). Following a previous paper,! 
continued use is made of the statistical properties of the function x(¢) 
and the output after (tf) is infinitely clipped. Under the assumption 
that a given axis-crossing interval is independent of the sum of the 
previous (2m -++ 2) intervals, where m takes on all values, m =0, 
1, 2, °°: , an integral equation is derived for the probability density 
P,(r) of axis-crossing intervals. This equation is solved numerically 
for several examples of Gaussian noise. The results of this calcu- 
lation compare favorably with experiment when the high-frequency 
cutoff is not extremely sharp. Under the assumption that the suc- 
cessive axis-crossing intervals form a Markoff chain in the wide 
sense, infinite integrals are found which yield the variance o?(7) and 
the correlation coefficient «x between the lengths of two successive 
axis-crossing intervals. These parameters are obtained numerically 
for several examples of Gaussian noise. For bandwidths at least as 
small as the mean frequency, « is large. For low-pass spectra, «x is 
small, yet the statistical dependence between successive intervals 
may be strong even when the correlation «x is nearly zero. 


INTRODUCTION 


of the distribution of intervals between axis 
crossings in random noise. In a previous paper,’ 
the author discussed this distribution in terms of the 
autocorrelation function of infinitely clipped noise, 
without assuming that the original noise was Gaussian. 


ers PROBLEM treated here is the determination 


* Manuscript received by the PGIT, July 8, 1957. 
+ School of Elec. Eng., Purdue University, Lafayette, Ind. 
Formerly with U. 8. Naval Ordnance Lab. ., Silver Spring, Md. 
1J. A. McFadden, ‘“The axis-crossing intervals of random func- 
tions,’ IRE Trans. on Inrormation Turory, vol. 2, pp. 146-150; 
December, 1956. Hereafter, this reference will be denoted by the 
symbol (1). 


This general discussion is continued now, and some new 
specific results are presented for the Gaussian case. 

Wherever possible, the mathematical notation is the 
same as in the previous paper (1). The only exception 
is the use of the symbol P,(r). 

In the next section, several infinite series are derived 
which involve P,(7), the probability density of the sum 
of (n + 1) successive axis-crossing intervals. Following 
that, the initial behavior of Po(7) is discussed, particularly 
in the Gaussian case. 

Next under consideration is the probability density 
P(r) when successive axis-crossing intervals are in- 
dependent, and again under a slightly weaker assumption, 
which is called quasi-independence. In either case, an 
integral equation must be solved numerically. 

The last section solves for the variance of axis-crossing 
intervals and for the correlation coefficient between two 
successive intervals, under the assumption that the suc- 
cessive intervals form a Markoff chain in the wide sense. 

Appendix I derives the general relation between 
P,(r) and p(n, r), the probability that a given time 
interval of length 7 contains exactly n zeros. 


DERIVATION OF THE Basic EQuaTIONS 


Some infinite series will be derived now which will 
prove useful in the subsequent sections. 

Let &(¢) describe a random process which is both 
stationary and ergodic. Assume that the mean value 


(é(t)| = O and that the probability density of &(¢) is 
mmetric about the mean. The normalized autocorrela- 
on function of £(t) is denoted by p(r). 

Let the function x(t) be the output after &(¢) is infinitely 
ipped. Then 


ot) ts ah et) "0; 
| au al itt) <0: 


The autocorrelation function of x(t) is denoted by r(r)’ 
the function x(t) is of interest because the axis crossings 
re preserved in the clipping process. 

Now consider an interval of time (¢, ¢ + 7). Let p(n, 7) 
the probability of finding exactly n zeros in the given 


(1) 


hterval. If the number of zeros is even, then (and only 

en) x(t) and x(t + 7) have the same sign. Then the 
lutocorrelation r(r), or Elx(t)a(t + 7)], is given by the 
hfinite series,” 


AG) 2d (—1)"p(, 7). (2) 
Now define the function P,(r). P,(r)dr is the con- 
litional probability that the (n + 1)st zero after time t¢ 
iccurs between ¢ + 7 and ¢ + 7 + dr, given a zero at 
‘me ¢t. In other words, P,,(r) is the probability density of 
ne interval between the mth and (m + n + 1)st zeros, 
there m is arbitrary. Po(7) is the probability density of 
tervals between successive zeros, which was denoted 
imply by P(r) in (1). 
The following identity between p(n, r) and P,(r) is 
roved in Appendix I. If n > 2, 


p”’ (n, 7) = BIP»(7) — 2P,-1(7) + P,-2(7)], (3) 


rhere 8 is the expected number of zeros per unit time. 
forn = 0 and 1, 


p’'(0, t) = BPo(7); (4) 
pl, 7) = BIP,G) — 2Po(s)]. (5) 
ba. (4) has been given previously by Kohlenberg.’ 
Now if (2) is differentiated twice with respect to + 


provided the second derivatives exist), and (3)—(5) are 
ibstituted, the following series is obtained in P,(r): 


3 (—1)"P, (0). 6) 


n=0 


yr’! (7) e 
| 46 
If Ar) Is neglected when 7 is small and-n > 1, then 
(7) & r’(r)/48. This approximation agrees with 
25) in (1). Special cases, in which this approximation 
5 not valid, will be treated later. 
Another infinite series involving P,,(7) may be derived 
so. Let U(r)dr be the conditional probability that a zero 
curs between t + 7 and t + 7 + dr, given a zero at 


2S. O. Rice, “Mathematical analysis of random noise,” Bell 
ys. Tech. J., vol. 23, pp. 282-332; July, 1944, and vol. 24, pp. 46- 
56; January, 1945, see (2.7—2). ec ghee : 

3 A. Kohlenberg, “Notes on the Zero Distribution’ of Gaussian 
oise,”’ M. I. T., Lincoln Lab., Lexington, Mass., Tech. Memo. 
L, p. 4; October, 1953. 
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time ¢. For Gaussian noise, U(r)dr has been derived by 
Rice.* The infinite series is 


UG) = DP. (7) 


The interpretation is relatively simple. If there is a zero 
in (¢ + 7,t + 7 + dr), it must be the first, or the second, 
or in general, the (n + 1)st zero after the one at t, where 
n 1S some nonnegative integer. 

If P,(7) is neglected when 7 is small and n > 1, then 
Po(r) = U(r). A better approximation than this one 
land better than that obtained from the first term of 
(6)] can be found by adding (6) and (7). Then 


eee PACA SE = 
5 ie He i | = 2 Pon(7) = Q(r). (8) 


Q(r)dr is the conditional probability’ that a downward 
crossing occurs between ¢ + 7 and t + 7 + dr, given an 
upward crossing at ¢t. Rice has suggested Q(r) as an 
approximation to Po(7) when 7 is not too large. For a 
given value of 7, Q(r) is obviously a better approximation 
than either U(r) or r’’(r)/48, since in (8) we need not 
neglect P(r) but only P2(r), Pa(r), ete. 

Still another series results if (6) is subtracted from 
(7). Then 


; Ee = (3) ey 22 Pas (9) 
For small 7 the left member provides an approximation 
to P,(r). 

The foregoing series in P,(r) is expressed in terms of 
r’’(r), B, and U(r). r’’(r) is the second derivative of the 
autocorrelation function of a(t), the clipped waveform. 
r’'(r) is also proportional to the autocorrelation of 2’(é), 
the waveform after infinite clipping and differentiation. 
Furthermore, by (12) of (1), 6 is equal to — 4r’(0 +). 

The function U(r) can be related to another statistical 
property of the clipped signal, namely, the fourth moment, 


w(71, 72, 73) = E[x(fatt + m)a(t + 72)a(t + 13)], (10) 
by means of the relation 

: 1 aw 

ha) bs 46 (52 aa\e) oD 


This result is derived in Appendix II. 


InrrIAL BEHAVIOR OF P(r) 


It was shown in (I) that when r(7) is nearly linear for 
small 7, the initial behavior of the probability density 
is Po(r) = r’’ (7) /48, and that this approximation becomes 
better as r — 0. This section considers the initial value 
P,.(0), particularly when r’’(0 +) # 0. 


4 Rice, op. cit., (3.4-10). : ; 
5 This function is discussed in (1). For Gaussian noise, Q(r)dr is 
given by Rice, op. cit., (3.4-1). 
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probability p(n, 7) that an interval 
contains exactly n zeros. By normalization, 


x PRT) = 


n=0 


Consider the 


(t, ¢ + 7) 
(12) 


Furthermore, the expected number of zeros in (t, ¢ + 7) 
is Br; therefore 


foe} 


= mon, 7) = Br: 


n=1 


A third equation in p(n, 7) is the series (2) for the 
autocorrelation function r(7). 

Still another series involving these functions p(n, 7) 
can be written for the quantity 26U(r). If n(r) is the 
number of zeros observed in an interval (¢, ¢ + 7), then 
the mean square is 
= n'p(n, 7). (14) 
If we differentiate twice with respect to r and p’’(n, 7) 
is eliminated by means of (3) and (5), then it is found that 


E[n*(7)] 


 bei(a)] = 28 D PAC, (15) 
or by (7), 
E” [n(x] = 28U(s). (16) 


For Gaussian noise, this result agrees with the equations 
of Steinberg et al.,’ who expressed E[n’(r)] in terms of a 
single definite integral. 

If (12), (13), and (2) are differentiated twice with 
respect to 7, and (16) is rewritten, we have the following 
four equations in p’’(n, 7): 


0= Dvn, 2), 


n=0 


foo} 


f= p> np’'(n, 7), (17) 
Gee a. 
28U(7) = Qi n'p’(n, 2). 


Suppose that the functions p(n, 7) are regular functions 
of s when r > 0. Then they may be expanded in a series 
about 7 = 0, 


2 


p(n, 7) = p(n, 0) + p’(n, 0)r + p’’(n, 0) S a 


In the case of p(0, 7) the leading term is unity, since it is 
certain that an interval of zero length contains no zeros. 
For pC, 7) the leading term is Br, for p(1, r) — Bdr as 
7 — dr. The leading term of p(2, 7) is of order 0 (r’), since 


6H. Steinberg, P. M. Schultheiss, C. A. Wogrin, and F. Zweig, 
“Short-time frequency measurement of narrow-band random signals 
by means of a zero-counting process,’ J. Appl. Phys., vol. 26, pp. 
195-201; February, 1955. 


(13). 


‘ 
M orem 


p(2, r) must become negligible compared to p(J, 7) as. 
tT — dr. ) 

Now suppose that the leading term of p(8, 7) is of or der 
0 (7°). More specifically, assume that p’’(n, 0) = 0 when 
n > 3. Then when 7 = 0 in (17), we have four equations 
in three unknowns, p’’(0, 0), p’(1, 0), and p’’(2, 0). 
These equations do not possess a consistent solution unless 
U(0) = r’’(0)/46. As will be seen, this restriction is not 
always satisfied. 

If the conditions are relaxed slightly, and it is assumed 
that p’’(n, 0) = 0 when n > 4, then when + = 0 in (17) 
we have four equations in {oue unknowns. The solutions 
are 


p’(0, 0) = 38U(0) + 3°"); 

DUS OS a8 UO oar 

p’'(2, 0) = —38U0) + 3"); (18) 
p'"(3, 0) = 38U@) — #0). 


The initial values of the probability densities P,,(7) may 
be obtained by using (81) of Appendix I. 


PO) = 5 | vo aj BO]; 
Po) = 4] vo - O], (19) 
PO) =.0, n> 2. 


Thus, under the given assumptions on p’’(n, 0), Rice’s’ 
approximation Q(r) for Po(r) becomes exact as 7 —> 0._ 
If UO) = r’(0)/48, then P,(O) U(0O) = r”’(O)/4e 
and? P7(0) 20; 

Specialize now to the Gaussian process. If &() is’ 
Gaussian and the normalized autocorrelation function is 
p(r), then Rice* has shown that 


u =? (1 + tan nye, (20) 
where 
7! = (2/0) Ma ee 
Hy = MxM es eedee 
M2. = —p’(0)(1 — p’) — p”, (21) 
Mas = p!"(1 — p°) + pp”, 
B = [—p’"(0)]'?/r. 


the dependence on 7 being understood. 

The autocorrelation function p(z) is symmetric with 
respect to 7 = 0. If it is regular at 7 = 0, then it can be 
expanded in a power series containing only even powers, 


p(t) = 1 —ar +er'4+.--- 
Using Rice’s formulas, it is found that r’’(0) = 0 and 


(22) 


Zale = 
2 (Oe tan H) 1 


| 
| 
(958 


s7— 0. Then U() = 0, and by (19), P.(0) = 0 and 
>(0) = 0. The initial behavior of P(r) and P,(r) in 
his (regular) case has been discussed more fully by 
Palmer.” 

Suppose, however, that p(r) is not regular. If at 7 = 0 
ll the derivatives of p(r) exist up through the fourth, 
hen the previous analysis will still apply. In terms of the 
ower spectrum W(f), where f is the frequency, it may be 
oncluded that P,(0) = 0 if the integral 


[ wor ar 


xists. This result follows from the Wiener-Khintchine 
heorem.° 

Suppose this integral does not exist. If p’’(0) does not 
xist either, then @ is infinite, as in the case when £(f) is 
lhe output from an RC low-pass filter and the input is 
vhite noise.” If W(f) behaves asymptotically like f~“* 
sf — , then p’’(0) exists, but p’’’(r) is discontinuous 
t 7 = 0. p(r) may be expanded in the series, 


pt) =1l—art+bf7y>+-::. (23) 
in this case r’’(0) > 0 and 

Per iil =i 2/3 1 

in2(h-+ ex) 22842 


‘hen U(0) > r’’(0)/46 and, by (19), P.(0) > O and 
(0) > 0. It was shown by Palmer’ that, when é(é) is 
qaussian and p(r) is of the form (22), successive axis- 
rossing intervals must be statistically dependent. Palmer 
howed that as t — 0, P(r) becomes asymptotically 
roportional to 7 and P,(r) behaves like r*, whereas if 
liccessive intervals were independent, then P,(7) would 
‘e a convolution of P,(7) with itself and therefore P,(7) 
rould behave initially like 7°. 

A similar analysis applies in the case of (23). It is seen 
hat P,(0) > 0 and P,(0) > 0, whereas if successive inter- 
‘als were independent the convolution integral would yield 
P,(r) which began at zero and behaved linearly for 
mall 7. 

The dependence between successive axis-crossing 
jes is discussed more fully in the following sections. 


APPROXIMATE SOLUTIONS FOR P,(r): 
STATISTICALLY INDEPENDENT INTERVALS 


This section discusses methods for determining the 
robability density P (7) and its moments under the 
ssumption that successive axis-crossing intervals are 
tatistically independent. 


| 


7D. S. Palmer, ‘‘Properties of random functions,” Proc. Cam- 
idge Phil. Soc., vol. 52, pp. 672-686; October, 1956. See pp. 679- 
0 


8 Rice, op. cit., (2.1-6). 

9Tn that case &(¢) is a Markoff process. Although 8 and Po(7) do 
t exist, there are other probabilities which can be calculated. See 
J. F. Siegert, “On the Roots of Markoffian Random Functions,” 
he RAND Corp., Santa Monica, Calif., Rep. RM-447; September, 
50. 
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Extensive use will be made of the Laplace transform 
y(s) of Y(z), 


y(s) = L{Y(n)} = i “TY (a) dr. (25) 
The following Laplace transforms are defined: 
Lir(/48) = 98); LLU} = uO; ep 
L{P,(7)} = prls). 
Transform the infinite series (6) and (7). 
1) = (1.0 @7) 
us) = Dp. (28) 


il 
oO 


n 


If it is presumed that successive axis-crossing intervals 
are statistically independent, then the probability densi- 
ties of sums of intervals are given by convolutions of 
P,(r), or in terms of the transforms, 


Prl8) = po''(8), (29) 

and (27) and (28) yield solutions’® for po(s). Respectively, 
pols) = gfs)/[1 — g(s)]; (30) 

pos) = u(s)/[1 + u(s)]. (31) 


These two solutions cannot be expected to be identical 
if the assumption of independence is incorrect. 

Equations related to (30) have been given by Lawson 
and Uhlenbeck" and also by Huggins.” The three random 
square-wave models of x(t) given in (I) (z.e., all but the 
case of clipped Gaussian noise) can be solved exactly 
by this equation. 

Eq. (31) is a basic equation in renewal theory.”” Miller 
and Freund’® have suggested its application to Gaussian 
noise and have solved it by assuming a simple function 
for U(r). They did not test their assumption (independ- 
ence) by a comparison with experiment. 

It is clear from the definition (25) that the moments of 
P,(r) may be obtained from derivatives of the transform 
Po(s), evaluated at s = 0. The easiest method is to expand 
the functions in power series, 


p(s) = 1 — E(r)s + E(r’)s’/2 —--- , 
g(s) = gO) + g’O)s + g’’(0)s°*/2 + --- , 


where / denotes the expected value, and substitute into 
(30). If r(7) and its derivatives vanish at infinity, then 
gO) === 10 =>) / 46. Bye (12). mal) or 0 eas, 


(32) 


10 Cf. M.S. Bartlett, “An Introduction to Stochastic Processes,” 
Cambridge University Press, Cambridge, Eng., p. 59; 1955. 

Also, W. Feller, “‘Probability Theory and its Applications,” John 
Wiley and Sons, Inc., New York, N. Y., ch. 12 and 13; 1950. 

J. L. Lawson and G. E. Uhlenbeck, ‘Threshold Signals,” 
McGraw-Hill Book Co., Inc., New York, N. Y., p. 45, (86b); 1950. 

zW. H. Huggins, “Signal-flow graphs and random signals,” 
Proc. IRE, vol. 45, pp. 74-86; January, 1957. See (48). 

13 J, Miller and J. EH. Freund, ‘Some Distribution Theory Con- 
nected with Gaussian Processes,’’ Virginia Polytech. Inst., Blacks- 
burg, Va., Dept. of Statist. Tech. Rep. 19, p. 129; May, 1956. 


18 IRE TRANSACTIONS ON INFORMATION THEORY 


therefore, g(0) = 
A/28, where 


>. Also g’(0) = — 1/46 and g’(0) = 


oe 


AY 2 if Aone (33) 
0 
The last two results, g’(0) and g’’(0), require integration 
by parts. It is assumed that tr(r) — 0 and 7’r’(r) > 0 
SUS eh are CON, 
Using the above series in (30), it is found, by matching 


coefhcients of equal powers of s, that E(7r) = 1/8 [in 
agreement with the discussion in (1)] and that 
E(7’) = 24/8 + 1/6". (34) 
Then the variance is H(r”) — [E(r)]’, or 
o(r) = 2A/p. (35) 


By the Wiener-Khintchine theorem, A is proportional 
to the spectral density (after clipping) at zero frequency. 
The spectrum of clipped Gaussian noise has been discussed 
by Van Vleck.** 

The renewal equation (31) may also be used for the 
computation of moments, but with some difficulty. The 
function u(s) does not exist at s = 0, since U(r) does not 
vanish at infinity. As 7 > ©, U(r)dr — Bdr since the 
probability of a zero in (¢ + 7, ¢ + 7 + dr), given a zero 
at t, becomes independent of 7. If 6 is subtracted and a 
new variable is defined, 


Vig) =U) = 8; (36) 
with a Laplace transform 
Lj Vin); = 08) = 2s) = 87s, (37) 


u(s) may be eliminated from (31) and v(s) expanded 
about s = 0. The results are that the mean is again 


E(r) = 1/6 and that the variance is 

a (ry = (=> 2B)/8% (38) 
where 

ps [ [U(r) = Bl dr. (39) 


It cannot be expected that the two results (35) and (88) 
will be equal if the assumption of independent intervals 
is incorrect. 

The function V(r) is proportional to what Bartlett” 
calls the ‘‘covariance density” of a stationary point 
process, when the points are the zeros of é(¢). 

For Gaussian noise, the solution for the probability 
density P(r) is not feasible by the Laplace transform 
method, since r’’(r) and U(z) are too complicated. Instead 
of (80) and (31), the two corresponding Volterra integral 
equations may be written, 


Po(r) = 1!’(r)/48 + [r’"(7)/48] * Po(z), 
Pir) = U(r) — Ua) * Po(n), 


(40) 
(41) 


14 T,awson and Uhlenbeck, op. cit., p. 57. 
15 Bartlett, op. cit., p. 166. 


March 
where (*) denotes the convolution process 


Vay SV (one / Y0¥eir—-Ddl. GO 
0 
These integral equations can be solved numerically, 
and, as stated before, the results generally will be different 
between (40) and (41). Palmer’® has solved (41) when 
p(r) = exp (— gr). 
QuaAsi-INDEPENDENT INTERVALS 


A new integral equation for P(r) is derived now. The 
equation is quite similar to (40) and (41), but the re- 
strictive assumption is somewhat weaker. 

Apply the Laplace transformation to the infinite 
series (8) and (9). Then 


Bul) + 9] = D Peal) (43) 
$lul) — 9] =D Pemnld. (44) 


Instead of assuming that all successive axis-crossing 
intervals are independent, now assume that a given 
interval is independent of the sum of the (2m + 2). 
intervals immediately preceding. This statement applies 
for all values of m, where m = 0, 1, 2, ---. This assump- 
tion will be called quasi-independence. Now Pom+1(r) 
is the probability density of the sum of (2m + 2) successive 
intervals. By our assumption, 


Pom-+1(8)Po(8) = Denials) (45) 
If both sides of (44) are multiplied by Pol) and the 


relation (45) is used, then p.(s), p(s), po(s), --- , may be 

eliminated by the use of (43). Then solve for a | 
8 ( 

p(s) = 5 (46) 


2+ uls) — g(s) 


Under this solution the mean interval is again H(7) = 1/8 
and the variance o(r) is the same as (38). In general, 
however, the solution Py(7) will be different from the 
solution ici either (40) or (41). 

As with (40) and (41), the best method for a practical 
solution is to write down the corresponding integral. 
equation, 


py =4[0@ +79 | 


-$| ua - nO | « Pn, (47), 


and to solve it numerically. Notice that the first term in 
the right member is Q(r), which Rice has given as an 
approximation to P (7) when 7 is not too large. The 
solution of (47) provides a correction to Rice’s approxi-- 
mation. 

For Gaussian noise, (47) has been solved numerically 
for several different power spectra. The normalized 


16 Palmer, op. cit., p. 678. 


| 
958 
utocorrelation function was obtained by the Wiener- 
<hintchine theorem; then U(r) and r’’(r)/48 were 
btained from (20) and (21). The convolution integral 
yas approximated by the trapezoidal rule, using a mesh 
vidth of Ar. Starting with P,(0) given by (19), we solved 
uccessively for P)(Ar), Po(2Ar), etc.’ The most time- 
onsuming part of the calculation is the computation of 
J(r) and not the solution of the integral equation. 

_ The following spectra have been considered. 

Case 1: 


ee ee (48) 
yhere m = 0, n = 2. In this case, 
p(r) = (1 + |rfjen'"!. (49) 
_ Case 2:m = 0, n = 3, and correspondingly 
os) = (1+ |r] + ger", (50) 
Case 3:m = 2,n = 4, and correspondingly 
ols) = (1+ [ol — 27? +4 [re (1) 


Case 4 is the spectrum of the output of a particular 
pw-pass Butterworth filter, if the input is white noise. 
vet 


ee 
“hen, i 2afo = 1, 


[os G7 fo) 12: (52) 


I) = = cin 5 fo" 
| 3 
oe 9 Decne cos E = |7| gin ns |\ (53) 


Case 5 is the ideal low-pass spectrum. 


WP 


constant, 


ORS i <a, 
fz fo: 


if 2rf. = 1, then the normalized autocorrelation function 
5 


(54) 


p(r) = (sin 7)/r. (55) 
| Case 6 is the ideal octave band-pass spectrum, 
Wf) = 0, Of = fo, 
Sr Meonsvant. 8h fo. J-< Zio, (56) 
= 0, Jz 2ho. 
C simplicity, again let 2rf) = 1. Then 
p(r) = (sin 27 — sin 7)/r. (57) 


: Cases 1 and 3 belong to the class where W(f) behaves 
ke f * as f — ~; therefore, the limit (24) must be used 


17 A more refined treatment is considered by L. Fox and E. T. 
oodwin, ‘The numerical solution of non-singular linear integral 
uations, ” Phil. Trans. Roy. Soc. London, vol. A245, pp. 501-534; 
bruary, 1953. See p. 522. ; 
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to determine U(0). In 
r'"(0)/48 = 

The solutions of (47) under these six conditions were 
obtained with the aid of an IBM 650 Magnetic Drum 
Calculator and are shown in Fig. 1. The first four are 
compared with the experimental results of Favreau, 
Low, and Pfeffer.’* The actual experimental points are 
shown when available. In Fig. 1(b) and 1(d), the agree- 
ment is very good. In Fig. 1(a) and 1(c), the agreement 
is very good also except near 7 = 0. These discrepancies 
may be caused by the absence of extremely high fre- 
quencies in the experimental measurement; 7.e., the actual 
spectrum may have deviated from the theoretical function 
Wf). In some cases, the solution Po(7) dips negative and 
therefore cannot be accepted for large intervals. 7. 
Evidently the assumption of quasi-independence is not 
valid here. 

In each part of Fig. 1 Rice’s function Q(r) also has 
been plotted. In those cases in which the upper frequency 
cutoff is extremely sharp (cases 5 and 6), it is found that 
Q(r) and the solution of (47) agree closely over most of 
the range of axis-crossing intervals. When they agree, it 
may be concluded that Q(r) is a good approximation to 
the true probability density Po(7). Beyond the range of 
agreement, the solution of (47) soon dips negative; 
therefore, the range of validity of the solution of (47) 
cannot be much greater than the range of applicability of 
Rice’s function Q(7). An extreme example is the ideal 
octave-band case 6. On the other hand, when the cutoff 
is gradual (cases 1-3), then Q(7) is not at all useful as 
an approximation to the true P,(7), but the solution of 
(47) is quite accurate. 

What can be concluded about the assumption leading 
to (47), 2.e., the assumption of quasi-independence? On 
the basis of these examples it appears that when the 
high-frequency cutoff is gradual, the assumption is good. 
When the cutoff is extremely sharp, the assumption is 
poor. In any case the assumption is not good for extremely 
large intervals 7 therefore; it should not be used for the 
calculation of moments. 

The assumption of independence also has been tested 
by the solution (not shown in Fig. 1) of (40) and (41). 
In case 2 the two results are extremely close to each 
other and to the curves in Fig. 1(b). Thus it appears 
that renewal theory can be used, at least approximately, 
for this type of spectrum. In all the other cases, however, 
the results are badly inconsistent and the assumption of 
independence should be rejected. 

The question of dependence is discussed again in the 
following section. 

Favreau, Low, and Pfeffer’* have suggested that for 
case 1 P(r) may be fitted exactly by an exponential 
curve. Since H(r) = 7, the initial value would be P,(0) = 
1/z, in conflict with (19), from which P,(0) = 0.40600. 


the other U0) = 


cases, 


18 R, R. Favreau, H. Low, and I. Pfeffer, “Evaluation of complex 
statistical functions by an analog computer,” 1956 IRE Nationau 
ConvrEntTION ReEcorRD, pt. 4, pp. 31-37. 
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(f) Ideal octave-band spectra. 


Fig. 1—The probability density Po(r) of axis-crossing intervals for six examples of Gaussian noise, under the assumption of quasi- 
independence, obtained by the solution of the integral equation (47). Also shown are Rice’s function Q(r) and its asymptotic limit, 


and the theoretical value of the mean interval, E(r) = 
and Pfeffer. 


1/8. The experimental results are taken from the work of Favreau, Low, 


‘hus, if the hypothesis is valid [p’’(n, 0) = 0 when n > 4] 
he solution for P(r) cannot be exactly exponential. 


) 


| Markorr CHAIN OF INTERVALS 

7 

This section shows the derivation of the variance of 
xis-crossing intervals’? and the correlation coefficient 
etween two successive intervals, under the assumption 
hat the successive intervals form a Markoff chain in the 
vide sense. 

The basic equations will again be (27) and (28), which 
avolve the infinite series in the transforms p,(s). The 
rst step is to express p,(s) as a function of p(s). Let 


Px(8) = An(8)po" '(8), (58) 


vhere a,,(s) has not yet been determined. This equation is 
generalization of (29), which followed when successive 
atervals were independent. This relation will be sub- 
tituted into (27) and (28) and the series summed in n. 
“hen the second moment in the expansion of po(s) will be 
dentified, as in (32). But first, a,(s) must be determined. 
If po(s) and p,(s) are expanded in powers of s, specifying 
he appropriate moments, then (58) will yield a series for 
_.(s). Now P,(r) is the probability density of the sum of 
n + 1) successive intervals, 7,, 72, °-* , Taxi, and the 
nean value of the sum is (n + 1)/6. Then from (58), 


8 Sie : n+1 
[1-24 $m - | 


1+ taL(E 2) ]- 


Sate pE)} + O(8').. (59) 


Eq. (59) can be simplified considerably if the mean 
uare of the sum of intervals can be expressed in terms 
the variance of a single interval. Use will be made of 
he correlation coefficient «;; between the 7th and jth 
i Saas intervals. Let 


ki = [E(ri7s) — V/8'Vo°, G7), (60) 


here o = o (r;) = o (7;), or simply o°(r). o°(r) is not 

ecessarily given by (85) or (88), since we have not 

ssumed independent intervals. 

Now the assumption is made that the successive axis- 

rossing intervals form a Markoff chain in the wide sense,” 
that 


kg ee”, (61) 


rhere «x is the correlation coefficient between two suc- 
essive intervals. Under this assumption, the mean square 


19 This result was first presented to the Institute of Mathematical 
atistics in Washington, D. C., March 7, 1957. See J. A. McFadden, 
he variance of zero-crossing intervals,” Ann. Math. Statist., vol. 
, p. 529; June, 1957. 

20 J. L. Doob, ‘‘Stochastic Processes,’’ John Wiley and Sons, Inc., 
ew York, N. Y., p. 233, (8.2); 1953. 
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of the sum of successive intervals is 


+ = [—2« + nl — «°) + 2k"). 
a) 
n may be replaced by (n + 1) and the result substituted 
into (59). For a,(s), 


kin — (n + De + 77] 
(ns 


(62) 


a,(s) = 1+ os + O(s). (63) 
If « = O there is no dependence between successive 
intervals except possibly in the third and higher moments. 

Eq. (58) can be substituted now into the infinite series 
(27) and (28). In the case of (27), the series is summed 
in n, the functions p.(s) and g(s)- expanded as in (82), 
and finally, the coefficients of s” identified. Solving for o’, 


(64) 


If « = 0 we have the previous result (35). 

If (58) is substituted into the other infinite series (28), 
a singularity is found. However, if w(s) is eliminated by 
means of (37), the series summed in n, and the functions 
po(s) and v(s) expanded about s = 0, all singularities 
disappear. The coefficients of s’ may be matched again 
and solved for o”. Then 


Wire B lee 
cee hey 


If x = 0 we have the previous result (38). 

Now if the Markoff assumption is valid, the two results 
(64) and (65) may be equated and solved for « and o°(r). 
It is found that o’(r) is the geometric mean of the two 
previous results (35) and (38), 


= | 2A Lee 
mit oe 


and that the correlation coefficient « may be obtained 
from the ratio of these same results by 


1 Bye u Ge Re at. ae 
Le eee 8 B° 
i i. last equation, the positive root was chosen so that 

Pa eeu be 

If the two results (35) and (38) (based on the assump- 
tion of independent intervals) are equal, then (67) gives 
the result x = 0. 

a(r) and x have been computed for several examples of 
Gaussian noise. The integrals A and B were obtained 
numerically by Simpson’s rule on an IBM 650. For 
Gaussian noise, r(7) is given by (9) in (1): 


o (7) = (65) 


(66) 


(67) 


qoL= = sin nee (68) 


The results are given in Table I, together with the mean 
value H(r). 


TABLE I 


Man Axrs-Crossine Interval H(r), STANDARD DEVIATION o(7), 
AND CORRELATION COEFFICIENT k BETWEEN Two SUCCESSIVE 
INTERVALS FOR GAUSSIAN NOISE OF VARIOUS SPECTRA 


Spectrum Correlation 1 
Case W(f) p(r) E(r)==}| o(r) K 
(eq. no.) (eq. no.) B 
1 (48), m=0,n=2 (49) 3.1416 | 3.22 | 0.058 
Deas): ), m=0, n=3 (50) 5.4414 | 4.68 | 0.005 
3 (48), m=2, n=A4 (51) 1.4050 | 0.775 | 0.486 
5 | (54) ideal (55) 5.4414 | 3.74 | 0.009 
low-pass 
6 | (56) ideal (57) 2.0567 | 0.565 | 0.442 
octave band 


As might be expected, the ratio o(r)/H(r) is smallest 
in the two cases 3 and 6 where the bandwidth is rela- 
tively narrow, 7.e., comparable to the midband frequency. 
These are also the cases in which successive axis-crossing 
intervals are highly correlated. In the low-pass cases 
1, 2, and 5, the correlation x is small. 

What can be said about the true value of the correlation 
coefficient between two successive intervals, without 
restoring to the Markoff hypothesis? Suppose it is assumed 
that « = 0. Then the expressions for variance (64) and 
(65) degenerate to (35) and (88), respectively. If the 
numerical results of these two formulas agree closely, 
then the assumption x = 0 cannot be far wrong. The true 
value of the correlation coefficient must be quite small, 
although not necessarily equal to the value of « obtained 
from (67). 

On the other hand, if the numerical results of (35) and 
38) differ widely, then the assumption x = 0 must be 
rejected. The true correlation must be large, although 
the actual value may differ considerably from « [obtained 
from (67)], depending on the suitability of the Markoff 
assumption. 

The ideal low-pass spectrum (54) is particularly sig- 
nificant. It was shown by a numerical solution of (40) that 
the assumption of independent intervals is far from 
correct, since the solution Po(r) dips far below the axis, 
much more than in Fig. 1(e). Yet the correlation x is 
nearly zero. Evidently the dependence between suc- 
cessive intervals is contained largely in the moments 
higher than the second. On the other hand, for a low-pass 
spectrum such as (48) (m = 0, n = 3), where the cutoff is 
gradual, it may be concluded not only that « is nearly 
zero but also that successive intervals are nearly independ- 
ent, since (by another calculation) the solutions of (40) 
and (41) appeared quite close to that of (47) and to the 
experimental result. 


CONCLUSION 


The results of this paper and the previous paper (1) 
may be summarized as follows. 


1) For a given process é(¢), the analysis of the clipped 
signal x(t) aids the understanding of the axis-crossing 
problem. 
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2) For Gaussian noise, the assumption of statistically 
independent intervals is usually not valid. The assumption 
of quasi-independence is a fairly good one when the 
high-frequency cutoff is not extremely sharp. This as- 
sumption should not be used to calculate moments. 

3) The mean value H(r) = 1/8 can be obtained from 
(12) in (1), v.e., 8 = — 3r’(0 +). This formula reduces to 
Rice’s result in the Gaussian case. 

4) Under the assumption that successive intervals 
form a Markoff chain in the wide sense, the variance 
o (r) of the interval distribution can be found from (66), 
and the correlation coefficient x between successive inter-. 
vals can be found from (67). These calculations require 
the knowledge of the functions r(7) and U(z). For Gaussian 
noise these. functions are known and the parameters 
o°(r) and x can be obtained by numerical integration. The 
Markoff assumption has not been tested by experiment. 

5) In some examples of Gaussian noise, x is very small. 
but the statistical dependence between successive intervals | 
is nevertheless strong. 

; 


It is felt that the best hope for future progress in this” 
problem lies in the invention of simplified models for 
stochastic processes. These models would not conform 
precisely to actual noise signals but would permit analyt-— 
ical results. The models must not assume independence 
but must provide for dependence between successive} 
intervals. q 


/ 
i 


APPENDIX [| 


Tur RELATION BETWEEN p(n,7) AND P,,(r) 


Now, (3) will be derived. This identity applies not only 
to axis crossings but to more general stationary point 
processes.” It is not assumed that successive intervals 
are independent. 

Let p(n, 7) be the probability that a given interval 
(t, t + 7) contains exactly n points, and let P,(7) be the 
probability density of the interval between the mth and _ 
(m + n + 1)st points, where m is arbitrary. 6 is the 
expected number of points per unit time. 

Consider the following events #,, H., and EH: 

H,: There are exactly n points in the interval (¢, t + 7). 

H,: There are exactly n points in the interval (¢, 
t+7+ dr). 

,: There are exactly n points in (¢, ¢ + +) and there 
is one point in the adjacent interval (¢ + 7, ¢ + 7 + dr). 
The probability of more than one point in an infinitesimal 
interval is neglected. 

fi, can be divided into two alternatives: 

H,: There are exactly n points in (t, ¢ 2 7) and 
none in (t + 7,¢ + 7 + dr). 

E,: There are exactly (n — 1) points in (é, ¢ + 7) 
and there is one in (¢ + 7, t + + + dr). 

By the conservation of total probability, we may 
write two identities concerning the probabilities of these 
events. 


Pr {#,} = Pr {#3} + Pr {#,} (69) 

Bre jor Prey (70) 
btraction of these two equations yields the following: 
| sees Veey as Et ea, = Prak, |= Pr fe.) (71) 
_ Now let these four probabilities be expressed. Clearly, 
Pr {Hi} = p(n, 7); (72) 
Pr {£,} = p(n, tr + dz); (73) 


d the difference, except for higher-order terms of the 
aylor-series expansion, is 


Pr {£,} — Pr {#,} = p’(n, 7) dr. 


(74) 


The right member of (71) presents more difficulty. Let 
}vO more events be defined: 

E,: There are not more than n points in (¢, ¢ + 7) and 
here is one in (tf + 7, ¢ + 7 + dr). 

E;:; There are not more than (n — 1) points in (é, t + 7) 
nd there is one in (t + 7, ¢ + 7 + dr). 

_ The desired probability Pr {H;} is the difference 


Pr {Hy} = Pr {E,} — Pr {B,}. (75) 
low 
Pr {B,} = Bdr [ P,(D al (76) 
nd 
| Pr{E,} = 6dr f P,.(@ dl. (77) 


hese equations may be interpreted as follows: dr is the 
robability of finding one point in (¢ + 7, ¢ + 7 + dr). 
ooking backward in time, 


i ” PD) dl 


the conditional probability that the (n + 1)st point 
rior to the one in (¢ + 7, t + 7 + dr) occurred before the 
me t, given a point in (f + 7, t + 7 + dr). Then (76) 
the joint probability of finding one point in (¢ + 7, 
+ 7 + dr) and not more than n points in (t, t + 7). 
q. (77) has a similar interpretation. Then by (75), 


Pr (B,) = 8dr f[ P.O -P.aDldl. (78) 


y similar reasoning, 


Pr (Ey =pdr{ PD —P,Dldl. (79) 
. T 

hen if the probabilities (74), (78), and (79) are sub- 
ituted into identity (71) and divided by dr, the result is 


2) = 6 [P.O - PD + PAD] al. BO 


the second derivative p’’(n, 7) exists, (80) may be 
fferentiated with respect to 7; (3) is the result. 
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The previous derivation applies only when n > 2; 
however, it may be extended easily to the cases n = 0, 
1 provided that the interpretation P,(7) = 0, when 
k < 0, is made. The resulting equations are (4) and (5). 

Eq. (8) may be considered as a difference equation 
in P,(7). Its solution is 

PG 2 Gn ee ee (81) 
k=0 * 

One simple example of the relationship between p(n, 7) 
and P,(r) is the following: if the number of points in an 
interval (t, ¢ + 7) obeys the (discrete) Poisson distri- 
bution 

Ba) ep 

pn, 7) = ae ae (82) 

then the time interval between the mth and (m + n + 1)st 
points obeys the (continuous) gamma distribution 


na+1in 
8 


cae (83) 


Pr) = 


APPENDIx II 


Tue RELATION BETWEEN U(r) AND THE 
CLIpPpED WAVEFORM 


Now (11), which relates U(r) to a property of the 
clipped waveform, is derived. Let 7, = x(t), % = x(t + 7,), 
t, = £(L 72), fy = “lt + 73), where 0) <mpip te te 
The probability that 2, = — 1,7, = +1, 2, = + J, and 
t, = —-l,denoted:by Ps,.2,18 


i 


| eB are aa 6 


[1 Soe aes 
=P fos moa eae re w], (84) 


where r;; = E(x,7;) and w = H(2,2.%37,). This equation 
is derived in the same manner as (14) in (J). The moments 
EQ), Ea) EG), (Ea agy;,), “anda @a57,7,) are 
written in terms of the probabilities P,,.., P.+-, ete. 
Then, set H(1) = 1 and H(z;) = O and assume that 
E(x,2,;2,) = 0 (¢ < j < hk), and solve for P_,,_.” In the 
case of Gaussian noise, P_,,_ is an example of the ‘‘quad- 
rivariate normal integral,’’ which has not been solved in 
closed form except for special values of the correlation 
matrix.” 

The correlation coefficients in (84) may be replaced by 
the autocorrelation function of the appropriate time 
lags; @.g., Tis = r(7,). The next step is to let 7, = dt, 
tT = T, and r; = + + dr. Then P_,,_ becomes the joint 
probability of an upward crossing in (¢, ¢ + dt) and a 
downward crossing in (¢ + 7, t + + + dr). If we divide 
by 46dt, the probability of an upward crossing in (¢, ¢ + dé), 
the conditional probability Q(z)dr is obtained. Thus 


— 


21 For a detailed derivation in n variables, see J. A. McFadden, 
“Urn models of correlation and a comparison with the multivariate 
normal integral,” Ann. Math. Statist., vol. 26, pp. 478-489; Sep- 
tember, 1955. See sec. 6. 

22 Ibid, bibliography. 


ee he 


48 dt 


Q(r) dr = (85) 
In order to utilize this result, P_.,- must be expanded 
in a double Taylor series in dt and dr. All terms are 
retained up to the second order. By differentiating under 
the integral sign in the time average, 


W(T1, T2) oe lim 5 
L 
rf w(Ha(t + ri)a(t + r2)a(t + 73) dt, (86) 
Ais 
it is found that = 
Ow aw 
(#) () ren, 
a) =(% y) is> 
& O+,7,7+ <= OT; Ona “at (0+), (88) 
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and furthermore, 


wO0+,7, 7+) = 


(89) 


The result of the detailed expansion is that all terms of 


P_,,— cancel out except the one proportional to dtdr; 
then (85) yields the result 
Ow 


1 yr 
- 8B (52 v) a é |. 


Q(7) (90) 


By comparison of (8) and (90), the final result, which is— 


(11), is reached. 

By differentiation of (86), it can be shown that U(r) 
in proportional to the autocorrelation function of the 
product a(t)a’(t +). 
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Recursion Formulas for Growing Memory Digital Filters’ 


MARVIN BLUM? 


Summary—A growing memory digital filter is defined by con- 
sidering the input (y,_,)—output (Z,,) relationship in the form 

= Duo” Wum Yn—uy Mm = 0, 1, 2, --- where W.,, is the weight- 
ing sequence of a linear time varying digital filter. Contained herein 
are a derivation of an optimum growing memory smoothing and 
prediction filter in the least squares sense for polynomial input func- 
tions, (of degree = K) and a theorem on the class of time invariant 
sequence W,, which are solutions of a difference equation of finite 
order, and an application of the theorem to the synthesis of sampled 
correlated noise by digital processes, using recursion formulas. The 
recursion formulation represents a practical solution to the genera- 
tion of a correlated noise sequence on line during simulation studies 
on digital computers. 


INTRODUCTION 


N a previous paper by the author,” the solution of 
the fixed memory least squares filter, using recursion 
methods, was presented. This part of the paper is 

an extension of the derivation and so the notation will be 
written with a minimum of explanation. 

For the fixed memory filter operating over an interval 
(n — 1)T, 2.e., the n most recent data points, the output 


* Manuscript received by the PGIT, Maly 27, 1957; revised manu. 
script received, August 21, 1957. 

+ Convair, Div, of General Dynamics Corp., San Diego, Calif. 

1M. Blum, “Fixed memory least squares filters using recursion 
methods,” IRE Trans. on INForMATION Tuerory, vol, 3, pp. 178- 
182; September, 1957. 


of the filter at time m7’ can be written in the form 


= a CUaaaea 


The action of the filter is to take a sliding weight of the 
n most recent data points as m is increased by one. 

The output Z,, will be an errorless estimate for input 
polynomials up to and including degree K in the absence 


of noise. The desired output is taken as the Lth derivative © 


of the input evaluated at (m + a)T. The equation for 
the coefficients C,, is given in Appendix I by (47). The — 
functions &,,(w) are the orthogonal polynomials 2, 3 
and S(v, n) is the sum of squares given by (9). The 


quantity £”,, (n + a) is obtained from the more general — 


definition: 


ak 
duc E, n(U) = £(0). 


(2) 


Instead of considering a filter with a fixed memory, 


one can let n change by one for each new sample. Thus 
when three samples are available one may fit, say, a 


least squares straight line to the 3-data points and obtain — 


estimates from the 3-point curve fit. When another 
data point is sampled, one may now fit a least squares 


958 


traight line to the 4-data points and obtain estimates 
om the 4-point curve fit. This procedure may be repeated 
ith the next sample to obtain a 5-point curve fit and so 
. The property of such a filter is that the memory span 
rows with each new data point and the estimates are 
ore accurate because they are based on more data. 
his increase in accuracy is obtained provided the co- 
cients of the polynomial remain invariant over the 
terval being sampled. In the fixed memory filter one 
quires only that the function be representable by a 
-olynomial over the interval of prediction and memory 
an of the filter. The coefficients of the polynomial may 
ence from interval to interval provided the degree of 
lhe polynomial does not increase. 
The output of the growing arc filter can be written 
rom a modification of (1) as: 


= >. Coy, 
u=1 


ie the coefficients C,, are given by (47) of Appendix I. 
or the sake of simplicity the sampling interval 7 is 
aken equal to one. To obtain the value of Z, for T # 1, 
hultiply C, by | T~” |. 


jeer he: tiK eos — 48) 


‘Recursion FoRMULAS FOR GROWING Memory FILTERS 


| One procedure for obtaining a growing memory filter 
+ to compute for each value of the array of coefficients 
‘, and utilize (3). This may be practical if one is interested 
A relatively small values of n. One may precompute and 
lore the values and use them as the data are obtained. 
s time progresses and ” becomes larger, this procedure 
ecomes burdensome and so an alternate method using 
cursion procedures will be considered. For purposes of 
onvenience the time origin has been labeled 1 and the 
ata are taken from 1 to n. The time origin can be arbi- 
rarily set to correspond to the first acquisition of data 
vithout restricting the generality of the solution. 
Let the Wth moment be defined as 


2 yu)” 
en the Wth moment satisfies the recursion relationship 
M(W,n) = MW, (@ — 1) + wl)” 
| Ween 0 lee 2ees Ke (5) 

efine 


Ww-»v ge oy oy 


= M(W,n) (4) 


2= al SQ, A 
iL ex dee Oe +a) 


hen (see Appendix I) 


K 
= MW, noxa.w- (8) 
w=0 
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ReEcuRSION ForMULAS For £!”,,(w) AND S(v,n) 


In a paper by Allen’ it is shown that 
(oD | I (n? — in} E 
Oye DI = Year ©) 


and therefore satisfies the recursion formula. 


Sv, n) = 


Sw, ) = (ett ston =) eat oan ta) 
When v + 1/n < 1 then 
So, n) & Sb, — DiI Sig 2 ss tt) (11) 


A recursion formula for the orthogonal polynomials 
is given by Anderson and Hauseman’ as: 


*"(n’ — v’) 
Er) = Ern(WEr al) — SS gs Aw), 
4(4° — 1) aD 
v=0,1,2 
where 
fon = 1 (13) 
and 
im =@-@, a= 254 (14) 


The recursion formula is an identity in wu considered as a 
continuous variable — © < u S + o so that (12) may 
be differentiated on both sides giving 


d” d? sas 
duz fey 5) ea fo) duZ & E, n(W) ate Lee au Teal £, nu) 
CY ees wae Z > — 
Th 
% = Opt eee 
In particular when Z = W, u-= 0 
1 
ae 0) = = _ in a aaa fe ”(Q) ef We ax()) 
2/2 
- Fem) 8) 


and when Z = L, and u = n + a then 


gH alata) = CER D py 4a) + LEM t al) 
= = v’) (L) 
4(4y” — 1) ene 1, n(n ate a). (17) 


2F, E. Allen, “The general form of the orthogonal polynomials 
for simple series, with proofs of their simple properties,’ Proc. Roy. 
Soc., ee vol. 50, pp. 310-320; 1935. 

L. Anderson and E. E. Hauseman, “Tables of Orthogonal 
Rae Values Extended to N = 104, ” Agricultural Experi- 
ment Station, Iowa State College of Agricultural and Mechanical 
Arts, Ames, Towa, Res. Bull. 297; April, 1942. 
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Eq. (8) can be generalized in matrix form if one lets 
the parameter L = 0, 1, 2, 3 --- K. Then one may obtain 
an output consisting of the predicted value of the input 
variable at (n + a)T and all of its derivatives up to 
the Ath derivative at (n + a)T. 

In computing the estimate of the Ath derivative using 
the various recursion relationships, the terms required to 
obtain estimates of all the derivatives up to K are avail- 
able as intermediate results. 


A THErorEeM oN GrRowING Memory Dicitau FILTERS 


Let the input to a linear time invariant digital filter 
be the sequence x, and the output, the sequence y,,, then 
the relationship between the input and the output is 
given by: 
OS EEG (18) 

“= 
where W,, is the weighting sequence of the filter. Define 
a class of sequences W,, such that W,, is a solution to a 


linear difference equation with constant (d,) coefficients 
of order »’ and let this class be noted by H,,., €.g.,° 


n? 


Dd. DW 0, Oral (19) 
k=0 
where D* is the advance operator defined by 
D'{W..} = Ware ae. . 2 = ‘i C (20) 
Seth, ee 


Then the following theorem for the class W, « H,, will be 


shown. 
Theorem: If x, = Ofor u < 0, then for W, ¢ H,, 


Ups i SS Woes 
u=0 
v= —n', —n’+1,:--0,1,-> (21) 
n'—1 
ORS = > Ea dye x ae YuXvtn'—k (22) 
k=0 
where 
k 
Jt = SZ dn Was (23) 
u=0 


Corollary: If x, 4 0 for uw < 0 then the output obtained 
from the digital filter W,, « H,,, on the assumption x, = 0, 
u < 0, as given by (21), is in error by an amount given by 


Coe etre. = Dy) Wire (24) 


u=n'+vy+1 


Sais = Yn' +o 
The proof of the theorem is given by Appendix II. 


4H. M. James, N. B. Nichols, and R. 8. Phillips, “Theory of 
Servomechanisms,’’ M.J.T. Rad. Lab. Ser., McGraw-Hill Book Co., 
Inc., New York, N. Y. vol. 25; 1947. 

5 ©, Jordan, “Calculus of Finite Differences,’’ Chelsea Publishing 
Co., New York, N. Y.; 1950. 
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DERIVATION OF THE WEIGHTING SEQUENCE WH, 


Consider a continuous linear time invariant filter whose 
transfer function is given by* 


_ Ap) 
G(p) 


where H(p) is a polynomial of degree h, and g(p) is a 
polynomial of degree n’ with n’ > h. Then the correspond- 
ing impulse response is given by the inverse Laplace 
transform W(t) and is of the form 


Y(p) (25) 


a Sx Ge preter: 
Wai) = >> Os, #20 (26) 
k=1 Gx=1 Ge Fe 1)! 
where the p,, p2, -:: Pa are roots of G(p) = 0 and the 


root px is of multiplicity Sx. 
A corresponding digital filter is obtained from defining 
the weighting sequence as 


W, = TWutT), (27) 


Substituting (27) into (26) one may write the following 
difference equation on W,,: 


ts = 0d ee 


a 


[] {@-— 2)" }w, = 0 


j=1 
where 


LZ; =e j= 12,5. 


(28) 


(29) 


If Y(p) isa stable filter then real part of p; = R{p;} <0, { 


j=l 2a; 


Expanding (28) one sees the W,, e H,,. It is also evident — 
that the class of weighting sequences W,, ¢ H,:, is derived — 


from the continuous filters which are representable as the 
ratio of rational polynomials in p when expressed in terms. 


of their transfer function. 


Noist SYNTHESIS 


The problem considered is as follows: let the output of 


the continuous filter whose input is white noise be sampled 
every 7’ units. Simulate the entire process by digital 
means. 


X(t) WO 


Fig. 1—Continuous white noise passed through stable linear filter — 


Y(p) and sampled every 7’ seconds. 


Fig. 2—Sampled white noise 
ae ae output has the same statistical characteristics as 
y(mT’). 


The analog can be seen from Figs. 1 and 2 in which the 
combined effects of passing white noise through a filter 
and sampling every 7’ units is simulated digitally by 
operating on the uncorrelated sequence (spaced 7’ units 
apart) with the digital filter whose weighting sequence 
is W,,. 


passed through stable linear digital — 


| 


The synthesis procedure will be summarized and re- 
ursion equations of the form (22) will be derived for 
O cases. 


SYNTHESIS PROCEDURE 


1) Fix the exact form of the continuous filter Y(p). 
2) Determine the roots p; of G(p) = 0. 
3) Solve for the impulse response W(t) using the 
inverse Laplace transform of Y(p). 
4) Determine the form of the difference equation in 
which W, = TW(uT) satisfies from (28). 
5) Expand (28), to obtain the coefficients d,,,_,,. 
6) Compute W,_, from 3) and 4). 
7) Substitute 5) and 6) into (23) and then into (22). 


EXAMPLE I 


Bemple Pair of Complex Roots 
Let 


G 
(ae) (Diet Pa) 
where C has the dimensions ¢’, therefore a = 2 = n, 
Sf = 1, and S, =’ 1. Let the real p, = real p¥ < 0. 


(he impulse response of Y(p) is given by the inverse 
uaplace transform 


Y(p) = (30) 


| WO = sf apy’ 1) 
0 that 
pee Ge eer (32) 
— pit B 
where 
pi = a+ (38) 
pnd pt =a — jB. 
Phen by (27) 
iW, = o*™ sin uBT, Me LD. eee, (34) 
hd 
| Zao * = o**(cos. BY +: 7 sin BT), (35) 
| Yee * = 6" (cos BT — 7 sin’ BT). 
o determine the d; one forms 
D— Z,)(D— Z,.) = D’ — (4,4 Z)D + Z,2:, (36) 
= d,D’ + d4,D+ d 
o that from (385) and (86) one obtains 
de =o. = I, 
d, = (37) 


—2e*" cos a 


d os en 
o= 
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The unknown coefficients go and g, are obtained from 
(23) as: 


go = W, = 0,7 | 


én (38) 
Ws dy + Wo dy = Wi = S Goin BIe"*| 


I 


gi 


Substituting into (22) one obtains the desired recursion 
formula. 


whe, = (2q 008 BY )yt.. — yt +S qsin BL (2943) 
(39) 
V2, eal Oa ae, 


where, g = e*” and y* = x, = O form < 0. 


One may verify by substitution that the value of 
yx,, generated by the recursion formula is identical with 


2+% 


Vee = IS Writes (40) 
u=0 
on the assumption that xz, = 0 form <0,v = — 2, 
Se OG a ee Oe 
Second-Order Low-Pass Filter 
Let 
g 
VAG) ere = and 
ergot 
4, = oe = i (41) 
then 
(D — Z,)’ = D? — 2DZ, + Zi. 
From (41) one obtains 
d,=d, = | 
d, = —2q,; i; (42) 
dz49 = q 
The impulsive response of the filter is given by 
1 eee 1 
WY) =s—C Fe 
Qn} Jerse (BP — Pa) 
WO =Ce. 
Therefore, by (27) the weighting sequence is given by 
Wa 0 Fad. 
Using (23) one obtains 
Gites ome I (43) 
gi = d.W, + ad,Wo = W, — (CT) q) 
From (22) one obtains 
Yes = — (dos + Giy is) 1 Gote+e fF Gikons: (44) 


Substituting (42) and (48) into (44) one obtains the 
desired recursion formula 


Urre = +29y%1 — qy* me (CT tn (45) 


a ¢ 


Error ANALYSIS 


At this point it is required to distinguish between two 
conditions on the input noise. For condition A the noise 
is switched on at t = 0, and one may assume that x, = 0, 
u <0, or is interested in the solutions under this condition. 
For this case the output of the digital filter using the 
recursion theorem is an identity. One may consider 
unstable filters of the form (25). The mean square of the 
output noise will tend to infinity as u — © if Y(p) is 
unstable. 

For condition B the noise is considered to exist from 
the present time to minus infinity. The use of the recursion 
formula assumes the x, = 0, uw < 0. This assumption 
causes a transient error as given by (24). If Y(p) is a 
stable filter, then the transient error ¢,, approaches zero 
asm— o as 0 | Ca | where p is the root of G(p) whose 
magnitude is closest to zero. Similarly all the statistical 
properties of y* approach y,, as m — o. Such properties 
as the autocorrelation function and power spectrum of 
the sequence y* approach the same functions of the 
sequence y,, as 0 | e?7"| as m — . Thus to simulate the 
sampled output of a filter Y(p) for white noise input 
under condition B, it is required that Y(p) be stable and 
that one use the steady state output of the digital filter. 
Thus the process is started at m = 0; however, the 
representative noise sequence is not used until m = M, 
where M is selected to satisfy some accuracy criteria. 


GENERATING SAMPLED WHITE NOISE 


A white noise source is characterized by having a 
constant power spectrum over all frequencies (f). If the 
spectrum is so defined that 


[ AW at = 0 = BO) 


where o is the mean square of the noise. Then for white 
noise, o is infinite. For sampled noise the power spectrum 
is periodic in f with period 1/7. The mean square error 
for the sampled noise is given by 


c= f ajaQ) 


where* 


A a1| 2 4 > ROD) We 2nft | 


and 
ROOT) = Eouxisioi)- 
For uncorrelated sampled noise 
ROOT) = or, b=0 
= 0 b#0 
2 Ay = Gy = 2ler. 


Therefore, given a white noise source whose power 
spectrum constant amplitude vs frequency is G, the 
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simulation of the sampled white noise with sampling 
interval equal to 7 is obtained then with an uncorrelated 
random sequence x, with mean square given by G)/27’. 
A method of generating a pseudo random sequence which 
is uncorrelated, distributed normally and capable of 
being scaled to the desired mean square is given by 
Kahn.° This sequence is used as the input sequence fom 
(22). The output sequence y*,,, for n’ + v = M then has : 
the desired statistical properties. - 

The method is summarized in Appendix ITI. 


APPENDIX [ | 


The input-output relationship of the digital filter is” 
given by 


4 

Zn = DUC Ke aie (46) | 

u=1 | 

Since’ | 
K (L) 

ONT Nn, a) ee 25 Sty, n) ’ (47) | 

then | 
| 

n K (LY | 


u=1 v=L Sv, n) 
One may expand &,,,(w) about (w — @) and obtain the 
identity ' 


B= SMe, a) 

where ; 
: 

a=" + e: (50) | 


Let the quantity (wu — %)* be expanded in powers of w. | 
Then i 


w= a = Dur —a)'(%), (1 
b=0 
so that substituting (51) and (49) into (48) one obtains, 


Me Ee Oa er Gk es 
SPDR DOD () 


u=1 v=L a=0 b=0 S(v, n) 


(52) 


Let (a — b) = W, then collecting like powers of u, e.g., the 
coefficient of u”, one obtains: 


6 (a ae eee 
DP Yee a 


(53) 


6H. Kahn, “Application of Monte Carlo,’ Rand Corp., Santa 
Monica, Cali, Res. Memo. RM-1237-AEC; April 19, 1954. Re-_ 
vised, April 27, 1956. 
_™M. Blum, “An extension of the minimum mean square pre- 
diction theory for sampled input data,” IRE Trans. on INroRMA- 
TION THEORY, vol. 2, pp. 176-184; September, 1956. 


. o (—1)’*” ae 
= ee ie 
(—1)"" (W) 


Sa) 


Ss mn (u) 


n+1 


(54) 


Then defining 


n 


M(W,n) = dS yu” 


u=1 


(55) 


tnd 


Sl Bn OE @ + a) 
(L) atl ; 
PK.n,W = p> W! S(v, n) 


v 


(56) 


bne obtains 
K 
Liss = Ss M(W, Mbic», W- 


APPENDIX II 
Proor oF THEOREM 


_ It will be shown that the difference equation which 
Hefines the recursion procedure is the equivalent of 
computing 

eee Wer, (57) 

u=0 

provided the weighting sequence W, satisfies the same 
complimentary difference equation as the y* do. This 
condition holds for the class of functions to which the 


method is restricted and for the weighting sequence as 
defined in (26) and (27). 


Proof 
Dropping the prime on n’, let 


m=n+v. 


Then (22) may be written 


in = 2 Open te Dy GiXm—; (58) 
From (23) one has 
gi; = Sa ate (59) 
and from (18) one has, for xz, = 0, m < 0, 
Yn = yy We Merce (60) 


Substituting (60) and (59) into (58) one obtains 
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n-1 m—n+k 


aa: D> > GVW a eae 


k=0 u=0 


Yn = 


i 


+ Si nds, We ere, 


7=0 v=0 


(61) 


Let m 2 nandm — (n—1) = L,L 2 Oand 


m—-nt+k—-u=L-—b, (ei ac Doe Lig 


Since the smallest index of x of the second summation is 
Lm-(n-1) = Xz, then the coefficient of x,_, is given by 
the first double summation only and is 
n-1 
< YS OW Grantee aun = W m-(L-b)) (62) 
k=0 
since the W,, satisfy the same complimentary difference 
equation as the y*. The coefficients of the X, when u = L 
are given by the sum of the two terms of (61) as follows: 
Let (m—-n+k—u)=L+e,c=0,1,2,3,---,m— L, 
then the coefficient of x,,, from the first term of (61) is 
given by 
al 
(—) Ss dW ome baLees (63) 
k=0 
and the coefficient of x,1,.) from the second term of 
(61) is given by , 


m—(L+ce) 
Cee W ACL fea oy: (64) 
v=0 ; 
Combining (63) and (64) one obtains 
m—(L+c) 
Uhte > Died wae ess 
»=0 
n—-1 
= oe ice = Gao newman (65) 
k=0 


Since all terms cancel by subtraction except the co- 
efficient of d, in the first summation, one will note that 
d, = 1. The combined results of (65) and (62) give the 
desired result 


pac DIVX 
u=0 


AppEnpDIx III 
GENERATION oF “SAMPLED” WuitE NoIsE*° 


Let Rr=-6), hy = Rex hor, — remainder ai ter 
dividing R; by 2°°, and let r; be scaled from 0 to 1. 

The approximate statistical properties of 7; are as 
follows: 

1) r; is a random variable uniformly distributed in the 

interval 0 to 1. 

2) E(r;) = 1/2 (average value of r;). 

3) Variance r; = E(r; — E(r,))? = 1/12. 

4) The r,’s are completely random, ¢.g., E(ritis;) = 
1/4 69; where 6;, is the Kronecker Delta, 6;, = 1, 7 = k, 
5, = 0,7 Hk. 

For most physical problems, the probability distri- 
bution of the noise is Gaussian. The probability dis- 
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tribution of 7; is not suitable for representing the noise 
encountered in physical systems. However, by a simple 
weighted addition of the r,;’s one may generate a variable 
N, which is almost Gaussian. In fact, by adding a sufficient 
number of 7,’s one may form a variable NV; whose dis- 
tribution is arbitrarily close to a normal distribution. 
This follows from the central limit theorem of statistics. 

Let 2%, vw, +--+ v, be L independent samples from a 
distribution f(z)dx and let the mean value of x be zero, 
the variance be unity, and 


In the limit as L — © the random variable Z is normally 
distributed with zero mean and unity variance. Consider 
the variable* 


N;= => Ginn — 1/2) 9. = 0,2 eo ee. 
b=0 


8 If the right-hand side is multiplied by \, then the variance will 
be 2. 


Note that the N,’s are formed from nonoverlapping sums | 
of the r,’s. Then in the limit, as L > ~, N, will be normally 
distributed with zero mean and unity variance. For all 
values of L, N; will have a zero mean and unity standard 
deviation. For L sufficiently large N; will be approximately y 
Gaussian distributed. 


PrRopEeRtTIES OF N; : 


1) FW,) = 0. 
2) Var (N;) = 1. | 
3) E(N;Nj+x) = 50, (completely random). | 
4) For large L, N; is approximately Gaussian dis- 
tributed. i 
5) N; is truncated since | N | < ~/3L and since the: 
standard deviation of N is 1. Then, if L = 6, N 
can vary between +4.2 standard deviations. It sf 
felt that 6 is probably a lower limit to L since, if N is | 
truncated too severely, the effects of occasional 
extreme deviations will be masked by the truncation | 
effect. | 


The Fluctuation Rate of the Chi Process” : 


RICHARD A. SILVERMANT 


Summary—tThe chi process is defined as a natural generaliza- 
tion of the chi distribution of statistical theory. A formula is derived 
for the expected number of level crossings per second of the: chi 
process. The formula contains as a special case the familiar expres- 
sion for the fluctuation rate of the envelope of Gaussian noise. 


INTRODUCTION 


HE expected number of times per second that a 
stationary random process &(t) assumes the value 
a is given by the familiar expression’ 


© 


Nee) = [| lvl pla,y) ay, (1) 


where p(x, y) is the bivariate probability density of &(¢) 
and its time derivative &(t), taken at any fixed time ¢. 
Consider first the case where £(t) denotes a zero-mean 
Gaussian process g(t), with correlation function 


* Manuscript received by the PGIT, August 14, 1957. The re- 
search reported in this article is supported by the Department of 
Defense under Contract No. DA49-170-sc-2253. 

+ Inst. Math. Sci., New York University, New York 3, N. Y. 

18. O. Rice, “Mathematical analysis of random noise,” reprinted 
in the collection “Selected Papers on Noise and Stochastic Pro- 
cesses’ (N. Wax, ed.), Dover Publications, Inc., New York, N. Y., 
p. 190; 1954. 


g(t + 7) 
Rr) = 

Ss akc (oatey 
Here the angular parentheses denote the expectation or 


ensemble average, and (w) is the (normalized) power 
spectrum of the process g(t). Then (1) reduces to” 


N,(a) a nee VA 0 W,(a), (2) 


where o” = (g°(t)), Rj’ is the second derivative’ of R(r) 
with respect to 7, evaluated at + = 0, and 


= |[ Bw) cos wr dw. 
0 


W(x) = 


i 
Vo ee 


is the univariate probability density* of the process” 
g(t), 2.e., the probability density of a Gaussian random — 
variable with mean zero and variance o°. Alternatively, 


2 Thid., p. 193. 

* The prime will be used to denote differentiation with respect 
to r, and the dot to denote differentiation with respect to t. We 
assume that g(t) and the components g,(t) of the chi process (see 
following) have correlation functions which are twice differentiable 
at the origin. 

* By the univariate probability density of a stationary random 
process é(¢), we mean the probability density of the random variable 
&(to), where ¢ is any fixed value of t. 
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) can be written as 


Nile) = [2 oVF Wile), (3) 


-Ri =a = | oe) do > 0 

0 

the mean square (angular) frequency of the process 
(t). 

Consider next the case where £(t) denotes the envelope 
A(t) of the Gaussian process g(t). Then, as shown by 
jeveral authors,’ ’ (1) reduces to 


Nala) = HE g Aw W.f(a), (4) 
where 
W.(x) = (x/o°) exp (=a /25 ); ce AN (5) 
W.(x) = 0 ’ i ANE 


s the univariate probability density of A(t), 7.e., the 
lamiliar Rayleigh distribution. The quantity Aw is given 
py 

(Aw)? = w — @)’, (6) 
where w is the mean square frequency of the underlying 
Jaussian process g(t), and 


a = 


ie wP(w) dw 


s its mean frequency. The similarity of (8) and (4) is 
| aes It follows at once from (3), (4), and (6) that 
ny two of the quantities o, N,(a) and N4(a) determine 
he other. 

In this note we define an infinite class of random 
rocesses, the so-called chi processes. We find that the 
xpression for the fluctuation rate of these processes is a 
atural generalization of (2), (3), and (4). 


Tur Cui Procsss AND Its FLuctTuATION RATE 


| We define the chi process by generalizing to random 
processes the familiar definition of the chi distribution.” 


@ 


Definition 
|. Let gi(t), +++ , Gn(t) be n independent stationary Gaussian 
random processes with zero means and such that 


(giQgilt + 1)) = o R(z), SS) S03 (7) 


88. O. Rice, ‘Properties of a sine wave plus random noise,”’ Bell 
Sys. Tech. J., vol. 24, pp. 109-157; January, 1948. See p. 125. 

6D. Middleton, “Spurious signals caused by noise in triggered 
circuits,’ J. Appl. Phys., vol. 19, pp. 817-830; September, 1948. 
See p. 828. 

7V. I. Bunimovich, ‘Amplitude peaks of random noise,’’ Zh. 
Tekh. Fiz., vol. 21, pp. 625-636; June, 1951. (In Russian. ) See p. 630. 

8 See, for example, H. Cramér, ‘Mathematical Methods of Sta- 
tistics,’’ Princeton University Press, Princeton, N. J., pp. 233-236; 
1946. 
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Then by the chi process with n degrees of freedom is meant 
the random process 


n L/2 
xn(t) = ie33 cto | (8) 

The processes g,(t), 1 <7 <n can be regarded as the 
components of a vector random process which has the 
process x,(t) as its length. This fact can be summarized 
by calling the processes g,(t), 1 <7 < n, the components 
of x,(t). The conditions (7) mean that the components of 
x,(t) are identically autocorrelated. The univariate dis- 
tribution of the process x,(¢) is, of course, the chi dis- 
tribution with n degrees of freedom, for which the 
probability density is* 


2 


W(x) = 2*5"T(n/2) ay 


"" exp (—27/20°), x >.0, 


We) 


I 
So 


; ag << (Y). 


[Note, the consistency of (5) and (9).] Processes of this 
type have also been studied by Miller, e¢ al., who call them 
generalized Rayleigh processes.°"*° 

We now derive an expression for the fluctuation rate 
of the chi process. 
Theorem 


The expected number of times per second that the process 
x,(t) assumes the value a is given by 


N,{@) = ie oV —Ro WA). (10) 


Proof 


We consider first the trivial but exceptional case n = 1, 
and note that xi() = V97(t) = | g(d) |. Since | g(t) | = aif 
and only if g(t) = a or g(t) = — a, it follows from (2) that 


N,(@) = Ae a V —Ri2W,,(o), ote (I 


N,(q) == 0 ’ 
Since by (9) 


a < Oe 


W,(a) = 2W (a), 
Wa) = 0 5 ane: 


(10) is established for the case n = 1. Note that unlike 
the case n > 2, x,(t) assumes the value zero (without 
passing through it) at a nonzero rate. 

When n > 2, we apply the basic formula (1), which 
requires that we first calculate the bivariate probability 
density of x,(¢) and its derivative x,(¢), taken at any _ 


a2 0; 


9K. S. Miller and R. I. Bernstein, ‘“‘Generalized Rayleigh pro- 
cesses” (abstract), Bull. Amer. Math. Soc., vol. 63, pp. 196-197; 
May, 1957. 

10K. §. Miller, R. I. Bernstein, and L. E. Blumenson, ‘“Gener- 
alized Rayleigh processes,’ Quart. Appl. Math., (to be published). 
These authors derive expressions for the bivariate and trivariate 
probability distributions of x,(¢), and for its correlation function. 
In some cases they allow the components g;(¢) to have means other 
than zero. 
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fixed t. We begin by writing the joint probability density 
of the 2n random variables g;(t), g;(t), 1 < 7 < n, @.e., 
the values of the components of x,(¢) and their derivatives, 
taken at any fixed ¢. It follows by the independence of 
the processes g;(f), 1 <7 < n, that 


(gi(D9i()) = (gi; 
and by differentiating (7) that 


(g:i()g.()) = o RO) = 0, 


where #’(0) vanishes by the evenness of the correlation 
function R(r). This means that the 2n jointly Gaussian 
random variables g;(t), g(t), 1 < 7 < n are uncorrelated 
and hence independent (being jointly Gaussian). More- 
over, we have 


(g(t) =0, gi) =o 
GO) = 0, Gi) = =e Re > 0, 


Thus, the joint probability density of the 2n independent 
Gaussian random variables g;(t), g;(f), 1 < 7 < nis” 


(11) 
1<ikj<n, 


PS BSW 


» Gn) 


. 2 
il Deis 


rT.) (Qr0)"(—Ri)”” exp 


PG, ap tks > Ons Gry ae 


dg 


2p// 
—2¢ Iie : 


rae 


where for notational simplicity the superfluous argument 
t has been omitted. 

We now introduce n-dimensional spherical coordinates 
by writing 


(== ASG SIs One SIN care 

Gp ==) Xn S100) Sil 6,“ eCOS 0.21 
ATT SE Pe ree ree hake pe (12) 
Gi OG SI. 6 CORE: ; 


Jn = Xn COS A, ; 


where x; = >> g;. The transformation (12) maps the 


1=1 


region — ~» <g; < ~,1 <7 < n into the region 


OPS Yas coy, 
0 eS 6,, Sey He) 6,,-2 om, 
OSS then <K Wars 


Since 


De G5 = Xm + xn + xn62 sin” 0 + ++ 
ee (13) 


2 A2 sone, Saas, 
== XnFn—1 sim 6, Sot eays LL Op= ay 


the joint probability density of the new random variables 
Xny 6, Abs ? Ons Nas 6,, bmi ats De 1S easily seen to be 


11Tn the remainder of this section we shall use the same symbol 
to denote a random variable and the dummy variable corresponding 
to it in probability density functions. This slight abuse of notation 
allows us to avoid the introduction of numerous additional symbols. 


March ° 
G2 i) 


P(Xn5 915° oe 5 Une Ox 


ee 
One CORE ye eos 


et xn Gi xn sin” OF + ++ tons sin” 01° 


“sin? i? 


—20° Ri 
agi, Joy °° * 5 Gn) Gis Gay Gn) 
x DE) 
O(Xn3 6,, ae) Oper, Xn) 6,, ees) 6,1) ; 


Since the partial derivatives of the variables gi, gs, --* , Gn | 
with respect to the variables x,, 6, «+: , 9,-1 vanish, the | 
Jacobian determinant in (14) reduces to 


| 
| 
| 


A(91, Goya Jn) HG, 92s eae » Gn) | 
O(Xn) 6, ate a Ces) O(Xn5 61, ete, 6,1) 
es | A915 Obj. BE? Gn) | Sf: 
— = J, 
OXn» 6,, Pk a 6) as) 


where the new Jacobian 


} 
Jn, = XE sin” “Opsin’ 7-6) = sineg es i 
is the square root of the product of the coefficients of the ; 
quantities x2, 67, --- , 6&_, in the sum (13). It follows by © 
integrating (14) with respect to the angle variables and 


\ 


their time derivatives that the bivariate eee 
; 


density of x, and x, is given by 
i 


) 


1 ile 
n T1\n/2 do 
Sha ose ene | 
ff d6,,_ al ape ie exp | -25 =o Ru 
heey _ Xabi ik Wa : 
Bs dé, exp [ ee ee dO, exp OG ea 
. / OL Oe, exp | - 


The integrals with respect to 6,, - 
immediately; their product is 


—2ro° Ri! (n-1)/2 1 
ae ere A —2 . = 
x, sin” 0) aint s Osea 


P(Xn5 Xn) an (Qra° 


x26?_, sin? 6, --- sin’ 6,_» 
— Ye ZR? 


-- , 6,-; can be done 


: 


sin’ 043 (16) 
(Sore Re = 
= i : 
Moreover, we have 
me ii do, - (17). 


aie 
f Ae te a 
“W) 


, the surface area of the n-dimensional sphere with — 
unit radius. Thus we find, using (9), (16), and (17), that 
(15) reduces to 


W(Xn) 


DP(Xny Xn) = 


\/ —2nRo’’o # —20°Ro J’ 
It follows from (18) that at any time ¢t, the process x,(t) 
and its derivative x,(f) are independent. Indeed, the 
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inivariate distribution of x,(¢) is Gaussian with mean 
ero and variance —oR{’, although its integral x,(¢) 
s manifestly non-Gaussian. This involves no contradiction 
nce x,(¢) is not a Gaussian process, 7.e., the joint distri- 
ution of its values taken at different times is not multi- 
‘ariate Gaussian. 

The proof of the theorem is completed by using (1) 
md (18), whence 


(os) 


Yule) = [| Xe | Pla Xa) A 


a 2W (a) x4 ° ex | Lee Xn =| d. * 
V —2nRi’o Jo orgs Sooke | 


p that 


Wie NE PV SRW a - ESD! © C0) 


In proving the theorem, the independence of the 
mocesses g(t), --- , g,(t) was used only to deduce the 
; lations (11). Thus we have the following result. 


! orollary 


| Let g:(t), --- , gn(t) be n stationary and jointly Gaussian 
andom processes which have zero means and satisfy (7). 
‘uppose in addition that the processes g,(t), --- , g,(t) are 
ot independent but are stationarily correlated, and that the 
r0ss correlations of gi(t), --- , gn(t) satisfy (11). Then the 
rpected number of times per second that the process 


x = [Soo |” 


ssumes the value a is given by (10). 

The difference between the process x,(¢) and the chi 
rocess x,(t) is, of course, that the components g;(¢) 
guring in the definition (19) are not independent. How- 
er, according to the corollary, which will be needed in 
e next section, the fluctuation rate of x,(¢) is also given 
y (10), provided that simultaneous values of the com- 
onents of x,(¢) and their derivatives are uncorrelated 
and hence independent since the g;(¢) are jointly Gaussian 
rocesses). On the other hand, independence of the 
omponents would imply the much stronger condition 


(gidgi(t’)) = g(Dgi)) = g.D9gi)) = 9, 
k<stiAjsn, 


| 
| 


(19) 


Dr all t and t’, not just fort = ¢’. 


| APPLICATION TO THE ENVELOPE FLUCTUATION RATE 


We shall now show that the general expression (10) 
icludes as a special case the formula (4) for the fluctuation 
pte of the envelope A(t) of a zero-mean stationary 
raussian Process g(t). First we recall the following facts 
bout the envelope.””’”” 


2 Rice, footnote 1, pp. 81-85. 

13 V, I. Bunimovich, “The fluctuating process as an oscillation 
ith random amplitude and phase,” Zh. Tekh. Fiz., vol. 19, pp. 
31-1259; November, 1949, (In Russian.) 
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1) For every w) > 0, there is a representation of g(é) 
of the form 


g(t) = g(t) cos wot — g,(t) sin wot, 


where the processes g,(t) and g,(t) are jointly Gaussian, 
and 


(ge(t)) = (g.(t)) = (g()) = 0, 
(gel) = (93) = (¢' @) = o°. 


2) The correlation properties of g,(¢) and g,(¢) are given 
by 


(g(ig(t + 7)) = (g.g.(t + 2)) = o R(x), 
(g-(t)g.(t + 7)) = —(g.(g.(t + 7)) = o S(r), 


(20) 


where 

Ro) = f "Ey con eee 
; (21) 
Sey = [ Hw) sin [wo — «)7] de, 


and &(w) is the power spectrum of the underlying Gaussian 
process g(t). The quantity 


k(r) = Bz) FS") (22) 


is independent of w . It is important to note that the 
processes g,(t) and g,(t) are independent only when S(z) 
vanishes, as it does, for example, when ®(w) is symmetric 
about wo. 

3) In particular, it follows from (20) and (21) that™* 


(get) g(t)) = So = 0, 
(gel) gs(t)) ra sae = 0, 


(23) 
and that 
(gel G.()) = —(Ge(Og.)) = So 

= fe (Oy oO aa ee 


where @ is the mean frequency of ®(w). 
4) The envelope A(t) is defined by 


AQ ig. Oo). 


and is independent of the choice of wo. 

The process A(¢) is not in general a chi process since 
the component processes g,(¢) and g,(t) are in general 
dependent. However, it follows from (23) and (24) that 
the conditions needed to apply the corollary of the 
fluctuation rate theorem are met, provided that we choose 
the frequency w», which is at our disposal, to be the mean 
frequency @, so that S/ vanishes. With this choice 


14'The subscript zero denotes evaluation at + = 0. 
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—Ri! = 


ie (a — Gy OO) doen 


v0 


where (Aw)” is the quantity defined by (6). Thus the 


envelope fluctuation rate N4(a@) is given by 


T 


Nala) = N,{a) = NE oV —RYW.la), 


March 
It can be shown that’® 


2E(k) — (1 — k’)K(k) — (r/2) 


Acar 2— @/2) : 


where K(k) and E(k) are the complete elliptic integrals 
of the first and second kind, respectively, and k is the 
function of 7 defined by (22). It follows that 


’? ae 2 te AYE J ey fe ey Oe 2s 2 2 | 

—R'x(0) = pa oe [2E(k) (1 kK) K(k) |nn1 (— BG 0) = ieer= (Aw)’, (25) 

where we have used formulas 112.01, 710.02, and 710.04 
: from Byrd and Friedman’s handbook of elliptic integrals.’° 
» Comparing (4) and (25); we see that another expression 

Nias az ope 4) sor N(a) es 


which completes the proof that (4) is a special case of 
(10). 

It is interesting to note another expression for N4(q@) in 
terms of the envelope correlation function 


(AMA + 7) — (AM). 
ZO AG) 


Rites = 


Loss of Signal Detectability in Band-Pass Limiters’ 


Nala) = Ae a ov —R';(0) W2(a). 


Tv 


15 See, for example, Bunimovich, footnote 13, p .1248. 

16 P, F, Byrd and M. D. Friedman, “Handbook of Elliptic Inte-} 
grals for Engineers and Physicists,’’ Lange, Maxwell, and Springer, 
Ltd., London, Eng., and New York, N. Y.; 1954. 
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Summary—A loss of signal detectability results from ideal 
symmetric limiting of a very narrow-band signal in narrow-band 
Gaussian noise. At small input snr this loss can be expressed in 
terms of the degradation of the effective signal energy-to-noise 
power per cycle ratio. Proceeding from results derived previously 
by Davenport and Price, an expression is derived for this degrada- 
tion in terms of the autocorrelation function of the input noise. The 
degradation is evaluated for three typical input noise autocorrelation 
functions and is found to be quite small in all these cases. It is 
seen that the degradation can be made to vanish by appropriately 
shaping the spectrum of the input noise. Regulation of the input 
noise spectrum to obtain conditions of limiter operation assumed 
in this paper may often prove to be a convenient method for reducing 
loss of signal detectability in band-pass limiters. 


in noise is frequently analyzed. in terms of a 

signal-to-noise (power) ratio as the significant 
parameter. Where fidelity of transmission is of concern 
the snr description has, of course, much to recommend 
it. However, when the presence of the signal is not a 
foregone conclusion and signal detectability is of primary 
concern, the snr description is no longer appropriate. 


| HE effect of nonlinear devices on a signal imbedded 


* Manuscript received by the PGIT, September 10, 1957. The 
research in this paper was supported jointly by the U.S. Army, 
Navy, and Air Force under contract with Mass. Inst. Tech. 

{ Lincoln Lab., M.I.T., Lexington, Mass. 


It has’ been shown that the effective detectability of 
an exactly known signal in the presence of additive white 
Gaussian noise is characterized by H/No, the ratio of the 
signal energy to the noise power per unit bandwidth.* 
Any irreversible operation (squaring, rectification, 
envelope detection, limiting) performed on the received 
message waveform usually destroys information and 
tends to decrease the detectability of the signal. This 
inherent loss in signal detectability can be characterized” 
by a decrease in the effective value of H/N > provided that: | 

1) The input signal-to-noise (power) ratio is small (so. 
that output spectrum is virtually the same whether or 
not the signal is present). 

2) The noise resulting from the nonlinear operation is” 
equivalent to white Gaussian noise over the region of 
interest. 

3) The resulting signal remains well defined. 

The important irreversible operation to be considered 
here is the symmetrical ideal band-pass limiter operated 
at low input snr (see Fig. 1). 


1W. W. Peterson, T. G. Birdsall, and W. C. Fox, “The theory 
of signal detectability,’ IRE Trans. on INroRMATION THEORY, 
vol. 4, pp. 171-212; September, 1954. 
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Fig. 1—Band-pass limiter. 


The output of the limiter, y(¢), is the following instan- 
aneous nonlinear function of the input, x(t). 


+1 if zx) >0 
Ht) = Osea (6) = Ore (1) 
—i sit a) <0 


he band-pass filter is designed to pass, without dis- 
rtion, only the spectral band of y(t) near the frequency 
the input. Spectral bands at harmonics of the input 
equency are rejected by this filter. The output of this 
nd-pass filter is denoted by 2(f). 

Davenport” has analyzed the effect of the band-pass 
iter on snr for a sine wave signal imbedded in narrow- 
and Gaussian noise. In particular, Davenport finds 
nat when the input snr, (S/N);,, is much less than unity, 
ne output signal is sinusoidal and the output. snr, 
S/N) out) iS given by ; 


out Sin 
te 


| S = — = 
| ( TN) oat Neos 4 Nee 


there Sin, Sout, Nin, Nout are the respective signal and 
oise powers at the input and output. It seems reasonable, 
hough rigorous justification is not given here, that 
avenport’s results hold also for the case of an arbitrary 
gnal imbedded in narrow-band Gaussian noise, provided 
at the signal bandwidth is sufficiently small compared 
the noise bandwidth. In particular it appears to be 
ue that the output signal is essentially an undistorted 
plica of the input signal. We shall use these results in 
e discussion which follows. 

The input waveform, x(t), is the sum of a noise wave- 
prm, n(t), and a signal waveform, s(¢), which may or 
vay not be present. 


nt) 
a(t) = or : (3) 
S(O an) 


‘he narrow-band Gaussian noise n(é) centered in the 
rane of the signal center frequency, denoted f,, has 
(two-sided) power spectrum depending on frequency f, 
enoted N;,(f). s(é) is assumed to be an exactly known, 
ery narrow-band signal with energy H;, given by 


| Eyx = He s(t)? dt. (4) 


rhe corresponding signal energy component of z(t) is 
noted L,,,, while the corresponding noise power spectrum 


2W. B. Davenport, Jr., ‘“Signal-to-noise ratios in band-pass 
iters,’ J. Appl. Phys., vol. 24, pp. 720-727; June, 1953. 
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component of z(t) is denoted N.u.(f). Nou(f) is approxi- 
mately the same whether the signal is present or absent 
because of the assumption that (S/N);, «1. 

If N;,(f) is approximately constant over the frequency 
band of s(t) so that it may be considered to constitute a 
white Gaussian noise background, the inherent detect- 
ability of the signal at the input is characterized by’ 


ON alf.) 6) 


(E/No)in = 


If Nou(f) is approximately constant over the frequency 
band of s(t) and if the bandwidth of s(¢) is sufficiently 
narrow compared to the bandwidth of n(¢), it appears, 
by virtue of the central limit theorem, that N,.,(f) is 
equivalent to a white Gaussian noise background over the 
bandwidth of the signal.* Therefore, the inherent de- 
tectability of the signal, given z(t), is characterized by 


| Oper, 
INGO. 6) 


The degradation factor, denoted A, which represents the 
inherent loss of signal detectability through the band-pass 
limiter is then given by 


(E/No)out = 


3 (E/No) in == EEN old) ; (7) 
(E/N o)ous TINGE) 
Noting from (2) that 
Ex San = (S/N) in in oe AN in (8) 
Bout 2 Seat - CS/N out our = WAN out 


we have for the degradation factor 


a Nin Nils) 3 
: TT N;n(f.) Nee (9) 


The main object of this paper is to evaluate A and its 
dependence on the shape of N;,(f). 

The band-pass limiter shown in Fig. 1 may be thought 
of as a device in which the limiter portion strips the 
amplitude variations from its input, while the subsequent 
filter, centered on the input frequency, is just broad 
enough to pass the input phase variations without dis- 
tortion. Thus, at low input snr, the output noise is essen- 
tially equal to the phase modulated component of the 
input noise. Following the terminology of Price,’ the 
correlation function of the input noise is written 


gx(7) = n(in(i + 7) = ¢,(O)o, cos [2rf,7 + X(7)] (10) 
where we identify 9,(0) = Ni,. f, is the (arbitrarily 


A 


3 The optimum detectability can be obtained with the aid of a 
correlation or matched-filter detector operating on x(t). These de- 
tectors achieve an effective snr equal to (2H/No)jn. For further 
details see Peterson, et al., op. cit. 

4 We are assuming here that the band-pass limiter is followed by 
a filter which is matched to s(¢). 

5 R. Price, ‘‘A note on the envelope and phase-modulated com- 
ponents of narrow-band Gaussian noise,” IRE Trans. on INror- 
MATION THEORY, vol. 1, pp. 9-13; September, 1955. Bunimovich 
obtained (5a) and (6) of this paper in 1949. 

V. I. Bunimovich, “Fluctuation processes as oscillations of 
random amplitude and phase,” Zh. Tekh. Fiz, (Leningrad), vol. 19, 
pp. 1231-1259; November, 1949. See (70) and (70a). 
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chosen) center frequency of N;,(f). o, represents the 
normalized amplitude of ¢g,(7) and satisfies oo = 1. A(z) 
is a phase modulating term which satisfies \(— 7) = 
— )(r) and vanishes when N;,(f) 1s locally symmetric 
about f,. From (5a), (5b), and (6) of the Price paper’ we 
see that the autocorrelation function of the output, 
ignoring the signal component, is given by 


g(t) = 2(t)e(t + 7) 


Tv 


a cos [2rf,7 + d(7)] 
: (3 Vets se) 
1+ 2 ha) | “ (11) 


where we identify 9.(0) = Nou. = 1/2. Using (10) and 
(11), introducing the necessary Fourier transformations 
and substituting in (9) we have 


ie o, cos [2rf,7 + Nz] |1 es aE 


[CGH rae |i 


March 


will actually yield a reduction in detectability loss. 
The proof that A approaches unity in the limit is given 
in the Appendix. 


UMW WAV, 


Fig. 2—An optimum noise spectrum. 


| 
1 


It is of interest to find the value of A for several typical 
shapes of N;,(f). For these cases we assume that N;,,(f) 
is locally symmetric about frequency f, so that \(r) = 0. 
It is further assumed that f, = f,. Then, since ¢, is slowly 
varying with respect to cos 47f,7, (12) reduces quite 
accurately to 


k\(k + 1)! i 


\e— 


ibs o, cos [2xf,7 + X(7)] cos Qrf,7 dr 


It is easily seen from the above equation that A is invariant 
to a change in 7 scale and hence to a change in the spectral 
frequency scale. 

It is apparent from (12) that the degradation in signal 
detectability comes about from the integral terms in the 
numerator involving 02". That each of these terms is 
positive can be seen by converting the Fourier transform 
to a multiple convolution of nonnegative spectral densities. 
Although o, cannot in general represent a true auto- 
correlation function having a nonnegative spectral 
density, o? must always be an autocorrelation function. 
This result is shown by taking two independent band- 
pass noises each with autocorrelation function ¢,(7). 
Multiplying them, we obtain a process whose autocorre- 
lation function is ¢2(7) and whose power spectrum in the 
region around de is seen to be the transform of 4 ¢2(0) 07, 
using (10). 

Often when limiting action is introduced, purposely 
or otherwise, into a receiver, the shape of N;,(f) can be 
regulated to some extent. The minimization of A with 
respect to N,,(f), subject to reasonable physical con- 
straints on N,,(f), does not appear to be capable of solution 
by the calculus of variations. However, consider the 
spectrum shape for N;,(f) shown in Fig. 2. The total 
input noise spectrum is the sum of two parts, A and B. 
Region B is wide enough to include the signal, and the 
height of A is negligible compared to that of B, but the 
power in B is negligible compared to that of A. With 
such a spectrum terms involving higher powers of og, in 
(12) are negligible and A = 1. This result is somewhat 
surprising, especially since it can be interpreted to mean 
that adding noise to the input, outszde the signal band 


oO i cos 2nf,7 dr 


AC roe ee 


14 = kik +31)! 0 


/ oat 

0 
si 3 be 5 

i a, dr s 3 [ c, He 
es) 64 fos) 

| o, dt / a, dr 
0 0 


¥ 7 
25 [ o, dr 
1024 2 
if o, at 
0 


Using this expression the following results are obtained: 

1) Rectangular N;,(f): For a rectangular N;,(f) which 
we take to be of width 1/7 since scale does not matter 
a, has the form 


1 
Sg 


oh Lo neal 


1+@ 


sin tr | 
on ie (14) 
Making use of the following integral,” 
© : 2k+1 
/ a “| es 
0 Tr 
T “ 2k ao 1 i fe 
= gta ; \en (Qh 2) ae 
k > 0 (ae 


* Bateman Manuscript. Project, “Tables of Integral Transforms,” 
McGraw-Hill Book Co., Inc., New York, N. Y., vol. 1, p. 20; 1954. 
See formula 11. 


s k+ . . . . . 
here (*"") is a binomial coefficient, we obtain for A 


alin ee 


po kk + 1)1(2k) 12” 


fea 


| ae a a Ve 1)(2k — 2j + 1)™ 


= 1c1.6: (16) 
When the signal is not centered in the middle of the 
ectangular band of input noise, A can be computed by 
sorting to (9), where Nu:(f,) is available from numerical 
ourier transforms of (11) performed by Price.° The 
sults show that A decreases slowly as f, approaches the 
and edge and is approximately 1.12 just short of the 
and edge. 

2) Gaussian shaped N,,,(f): Again choosing a convenient 
bale factor, ¢, has the form 


I! 
) 


Eo. (17) 


Cr eners) 


Pa Ge 1A 2k 


| 
‘iving for A 


at lt S: 


(18) 


| 
L, 3) Optically shaped WN;,,(f): Again choosing a con- 
Sa scale factor, o, has the form 


! ge, (19) 
‘ving for A 

IGE a 2b = 1) 
Haat y (a) alt e ’ 1.059. (20) 


his last case has the smallest degradation presumably 
ecause the power spectrum JN,,(f) comes closest in some 
nse to approximating the shape shown in Fig. 2. 
The most interesting and important conclusion to be 
oted from these results is that the degradation in de- 
ectability is quite small in all the cases considered. The 
esults suggest that appropriate regulation of N;,(f) to 
sbtain conditions of band-pass limiter operation assumed 
arlier, viz., (S/N)in < 1 and signal bandwidth much 
aller than input noise bandwidth, may often prove to 
ye a convenient means for reducing loss in signal de- 
ctability brought about by the limiter. 
In situations where detectability is not of primary 
eer snr is sometimes used to characterize the per- 
ormance of a limiter. Curves given by Davenport” for 
he band-pass limiter show that a two-fold increase in 
mr results when the limiter is operated at high input 
mr. One should not conclude from this result that the 
imiter should be operated at high input snr for best 
etection performance. As we have seen, it is possible to 
erate the limiter so that the inherent detectability of 
e signal is essentially unchanged even though the snr 
ecreases. SNR for a finite duration signal may sometimes 
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represent a lower bound on signal detectability, but this 
quantity says nothing about the inherent detectability 
which can be obtained with further filtering and conse- 
quent improvement in snr. 

The increase in snr obtained with the band-pass limiter 
is very similar to the increase in snr obtained from the 
band-pass envelope detector operated at high input snr’ 
(certainly a device which usually degrades signal detecta- 
ability). A factor of two increase in snr results in each 
case because the envelope detector suppresses the com- 
ponent of noise out of phase with the signal while the 
limiter suppresses the component of noise in phase with 
the signal. 


APPENDIX 


It is desired to prove that with the spectrum shape for 
N;,(f) shown in Fig. 2 all terms in the numerator of (12) 
involving o?" are negligible (so that A approaches unity) 
in the limit where the height of A is negligible compared 
to that of B, but the power in B is negligible compared 
to that of A. 

We denote the respective widths of regions A and B 
in Fig. 2 by W4 and W, and their respective heights by 
N, and Nz. Kq. (12) can be written 


(4 3) (2k=1)[ 
iy i) { a IT 
[ en) cos 2af.7 dr 


¢,(T)o," cos 2rf.7dr 


9 


pa 


(21) 


The denominator on the right will be recognized as simply 
3 N;,(f,). Since f, is always taken to lie in the region B, 
we have N;,(f;) 2 Nz». The series in the numerator above 
is dominated, term by term, by the following series 


1\(3\ ee 2h eee 
s ale) { 1) ; ) i C0) ide 


where the integral, being independent of k, may be 
factored out of the sum. The sum converges to a finite 
positive constant [which can be evaluated by setting 
rt = 0, ¢.(0) = 3 in (11)]. The o, and X(7) corresponding 
to a very narrow-band noise input are slowly varying 
compared to cos (27f,7). Therefore 


(22) 


[har = 46k [of ar. (23) 
From the above equations it follows that 
c / g(r) dr 

JOS aL Ss = Se 24 

= AO os 


where c represents a positive constant. By Parseval’s 
theorem, the integral in (24) can be converted to an 


78. O. Rice, ““Mathematical analysis of random noise,’’ Bell Sys. 
Tech. J., vol. 24, pp. 46-156; January, 1945. See sec. 3.10. 
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integral over V;,(f), and this yields 


[ g(t) dr = Ws. — Ws)Ni + 2Wz(N, + Nz)’. (25) 


Note also 


¢,(0) = 2WiNa + 2W2Nz. (26) 
Substituting (25) and (26) into (24), we have 
Ac te (Wa == Wp)Na ete W2(Na Sle Nz)’] 
ac N2(WANa« + WN) 
INA W al ls te) = 
< 
2 EE Lea ee cv 0 
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In order for the right-hand side to approach zero it is 
necessary and sufficient that 


ae n WN, — 0 (28) 
thus demonstrating the requirements on N;,,(f) stated 
earlier. However, these requirements can be met only 
insofar as bandwidth limitations will allow. 

Note that nothing in our analysis has precluded the— 
possibility of region A in Fig. 2 not overlapping region B 
containing the signal. This suggests the interesting 
possibility of using noise outside the band B of signal 
and noise both to control the limiting process and to 
reduce the loss in signal detectability. 


A Table of Bias Levels Useful in Radar 


Detection Problems* 


JAMES PACHARESt} 


Summary—A table of bias levels \ = \(m,p) corresponding to 
specified probabilities of false alarm, 10°”, when integrating over m 
independent noise pulses using a radar system with a square-law 
detector receiver, is given for p = 1(1)12 and m = 1(1)150. The 
notation ~ = 1(1)12 means that p varies from 1 to 12 in steps of 
unity. 


INTRODUCTION 


amplitudes, r, have the probability density 
function given by 


i SSUME that noise pulses result in signals whose 


@7°/2* (n /g?) ; (1) 


m 


of m independent such pulses, it is well known that 
t = R2/20° has the probability density function given by 


Then, if R represents the sum of squares of the amplitudes 


a nel (2) 


As a result of (1) the average value of r’ is 2c” and the 
average value of ris o-V1/2. 

If the bias level is set at c, then the probability of a 
false alarm equals the probability that R? exceeds c, 
which equals the probability that ¢ exceeds ¢/2o°. 

If 10 ” is the probability of false alarm when integrating 
over m independent noise pulses using a square-law 


* Manuscript received by the PGIT, July, 15, 1957. 
+ Systems Development Lab., Hughes Aircraft Co., Culver City, 
Calif. 


detector, then p, m, and X are related by the following - 
expression: 


io) eae ee 
il Cea (3) 


where ¢ = 20°d.* 

In practice one usually has a value of m and p in mind 
and would like to know the corresponding value of the 
bias level, X. ; 

Example: Suppose one wants the value of c¢ such that 
the probability of a false alarm will be one chance in a 
million when integrating over 25 independent noise 
pulses when o” = 2. From the tables we find that for 
m = 25and p = 6, = 56.3040. Consequently, ¢ = 20° = 
225.2160. 

Tables of the bias level \ =  (m, p) are given correctly | 
to four decimal places for p = 1(1)12 and m = 1(1)150. 
This table adequately covers the range which is likely to | 
be of interest in the above type situations. For m > 150 
one could use the asymptotic formula given by Wishart? — 
or the asymptotic formula (7), to find \. It should be noted — 
that since the values of \ given in the table are the prob-— 
ability points of the gamma distribution this table — 
is of use in solving many statistical problems in which 
the gamma distribution is used. 


‘J. I. Marcum, “A Statistical Theory of Target Detection by 
Pulsed Radar: Mathematical Appendix,”’ Project Rand, RM-753, 
p. 115 April 25, 1952: 

? J. Wishart, “x? probabilities for large numbers of degrees of 
freedom,” Biometrica, vol. 43, pts. 1-2, pp. 92-95; June, 1956. 


These are the only known tables in existence which 
over the above ranges in p. Pearson’s tables® give the 
alues corresponding to the integral (3) to seven decimals 
pr m up to 51 so that one cannot find the bias level ) if 
» = 7. Pearson’s tables not only do not cover the complete 
ange of interest but also, in situations where a point of 
nterest does fall within the range of the table, 7.e., if 
< 51 and p < 6, one would have to undergo the in- 
onvenience of using inverse interpolation to find 2. 
Tables have been computed by Teichroew* giving \ 

= 2(1)15,20(10)50, 100 and p up to 3 so that one 
ould not find the bias level \ from these tables if p > 3. 


MeruHop Usep To Comput THE TABLES 


Newton’s method was used to find the value of X 
satisfying the equation f(A) = 0, where 


jo) = [ ae - 107. 


if an initial guess, Ao, say, is available, then as is well 
known, Newton’s method yields a more accurate solution 
asing 


| X= Xo — f0e)/f/0). (4) 
The following relation was used to find f(A): 

| ro) e t Cia a at 
i Cs aie ae Cea 
where S(m, \) was computed from 


S(m, ») (5) 


Seay eee (m=) 


Since 


eon 
OM Senay 
®. using (5), becomes 
dh = Ao + S(m, ro) — A(m, ro) (6) 
where A(m, \.) was computed from 
log A(m, A) = A — (m — 1) logaA 
+ log (m — 1)! — p log 10. 


The first guess for A, Ao, was found using 


ho = m+ V Ql) + Que) + OY2 
$24. - BO 
m 


3K. Pearson, “Tables of the Ra Oc Gamma Function,” 
Cambridge Press, New York, N. Y., 1951. 

4D. Teichroew, “A table of millile probability points of the in- 
complete gamma distribution,’ Natl. Bur. Standards (U. S.); 
February, 1954. 
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where the Q’s are Campbell’s polynomials given by 


Qi(x) = « 


Q(t) = @ — 1/8 

Q(x) = (2° — 7x)/36 

Q:(x) = (—38a* — 7x” + 16)/810 
Q;(x) = (9x° + 2562° — 4332) /38880 


Qo(x) = (12x° — 243x* — 9232” + 1472) /204120 
Q,(a) = (—3753a" — 4353x° + 2895172" 
+ 289717x)/146966400 
Qs(x) = (2702° + 46142° — 9513x* 
— 104989x* + 35968) /55112400 


and where x is found from 
Bat /2 
Pee ae dt = 10”, 


using the literature.” Campbell’ derived the asymptotic 
expansion (7) from the relation 


awe 


fe an pi at = hee Ir dt. 


Only two iterations of Newton’s method were required 
on the average. 


CHECKING OF TABULATED VALUES 


Checks were made against the values given by Lewis’ 
for p = 3 and m = 20(5)50, 60 and showed that our 
results were all good to at least four decimals. Checks 
made against the values given by Teichroew* for p = 1, 
2, 3, and m = 3,50,100 again showed that our results 
were all good to at least four decimals. In addition, the 
table was differenced four times in both directions, in 
an effort to catch any obvious errors which might have 
occurred, with the result that no mistakes were apparent. 
Finally, several values were checked by hand computation 
to verify the results. Although \ was computed to seven 
decimals, only four decimals are given in the tables since 
the values were found, in the worst case, to be accurate 
to only four decimals. 
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| Summary—A modified form of pulse-code modulation, called 
weighted pcm, is described in which the relative amplitudes of the 
ulses within the pulse-code groups are adjusted so as to minimize 
the noise power in the reconstructed signal due to errors in trans- 
mission. A performance analysis shows the knee of the output signal- 
o-noise ratio curve to be moved 1.4 db to the left for a weighted 
even-digit pcm system. An information rate study reveals that the 
aximum improvement which can ever be achieved by any encoding 
rocess over a conventional seven-digit pcm system is only 8 db. 
he importance of selecting a suitable system worth criterion is 
emphasized by showing that weighting increases the information 
rate relative to an rms fidelity criterion but decreases it on a pure 
equivocation basis. 


INTRODUCTION 


| INCE its invention in 1939 [1] pulse-code modulation, 
S or pem, has received considerable attention because 

of its remarkable noise-cleaning property. The 
principles and practices of pem are well treated in the 
literature [2]-[5] and will not be reviewed here. It is merely 
necessary to note that the message signal is sampled 
periodically and that the samples are quantized to a set 
of discrete levels. The quantized samples are expressed 
as binary numbers and represented electrically as pulse 
sequences which are transmitted on the (noisy) channel 
to a receiver which reverses the procedure. 


* Manuscript received by the PGIT, July 15, 1957. This paper is 
based in part on the author’s doctoral dissertation at Northwestern 
University in 1953, and in part on work done subsequently at Moto- 
rola, Inc., and, The RAND Corp. The work undertaken at North- 
western was sponsored by the U.S. Army, Signal Corps Eng. Labs., 
Fort Monmouth, N. J., under Contract DA 36-039 SC-15375. 

+ The RAND Corp., Santa Monica, Calif. 


| Weighted PCM’ 


EDWARD BEDROSIANT 


A conventional pem transmission consists of equal- 
amplitude pulses; thus, the probability of error is the 
same for all the pulse positions. However, within a given 
pulse-code group it is obvious that the various pulses 
are of different importance to the reconstructed signal 
value since each pulse denotes the presence or absence of a 
different power of the binary base 2. It would appear 
that an improved performance could be obtained by 
‘weighting’ the various pulses, 7.e., adjusting their 
relative amplitudes, in such a way as to let the prob- 
ability of an error be in keeping with the value of the 
pulse. 


Tuer Error Noise PowrEr 


_ Let the pulse positions in a typical pulse-code group 
be denoted by the integers 1 to n and let the corresponding 
binary numbers 0 and 1 be denoted by positive and 
negative pulses, the amplitude of the 7th pulse being a;,. 
If the probability density of the signal amplitudes is 
uniform (or made so by compression), then the signal 
will occupy all quantization levels with equal likelihood 
and it can be reasoned that the mean value of the ensemble 
of pulse sequences will be zero. 

A simplified noise model is assumed wherein a random 
noise having a normal distribution of amplitudes is added 
linearly to the pulses. Also, the noise amplitudes at the 
various pulse positions are assumed to be uncorrelated. 
(This model corresponds quite closely to a receiver using 
synchronous detection.) The decision process can then 
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be evaluated by noting that the probability p; of making 
an error at the 7th pulse position can be written 


0,-= Ja | e*'”? dx, (1) 
us ar 


where the standard deviation of the noise, and hence the 
noise power, has been taken as unity. The average signal 
power becomes 


S= y a (2) 


since it is merely the mean-square pulse amplitude. 
Errors will be rare in an operating system so it will be 
assumed that p; «< 1 and that the occurrence of more 
than one error in a given pulse-code group can be dis- 
regarded. The probability that there will be just one error 
and that it will occur at the 7th pulse position is given by 


n 


pl) =p; [1 A — p) Spi. 


iol 
TA0 

Let /; denote the error which occurs in the reconstructed 

signal when there is an error at the zth pulse position. 

Since errors are equally likely to result from noise adding 

to or subtracting from the pulses and since the pulses 

denote binary digits, it follows that 


Beso) he ae 


The noise power N, in the output due to an error is the 
mean value of % over the pulse group, or 


N, = dS o(DE? = dS pai. (3) 
i SEL 


THe MINIMIZATION PROBLEM . 


The weighting function for the a; is found by minimizing 
the error noise power of (3) subject to the constraint: of 
constant power given by (2). Consider the function 


WG op) yas rae 


+ G1, to, °° 


where X is the Lagrange multiplier and where the variables 
of the function f are connected by the condition g = 0. 
By the method of Lagrange’s undetermined multipliers [6], 
the set of m + 1 equations in the 7; and \ given by 

oF 


Feros 


» Ung d) = f(a, Le oS 


ae) 


g=0 


is satisfied at the extreme values of f provided that not 


- all the derivatives 0g/dx; vanish. From (2) and (38) it is 
seen that 


f(a, Dep OS oy) =~ 


gar, Ce tas Oy = 
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Fig. 1—Comparison of weighted and unweighted pem systems. __ 
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SESE eS heii (4) 
N j=1 

It can be shown that a real, positive, unique solution to (4) | 

exists and yields a true minimum. The resulting weighting | 

| 

. 


function is well approximated by 


| 
a ) m 16 (5) | 


2 = 7—_—___— — 

a,;=S+ ines 1 
which is explicit and seen to depend only on the average | 
signal power and the number of digits per pulse-code — 
group. 


SYSTEM PERFORMANCE 


The compressed message signal was assumed to be 
distributed uniformly over the quantization levels. If 
the quantization steps are given unity amplitude the — 
output signal power becomes 


4" — 1 
12 (6) 


So = 


which is the resulting mean-square value. 

The quantization noise power is found by assuming 
that the quantization error amplitudes are uniformly 
distributed in all the quantization levels yielding 


+1/2 1 
N= | i pee (7) 
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Fig. 2—Weighting function for optimum performance at design 


input signal-to-noise ratio. 


The output signal-to-noise ratio then is written 


S So 

eu Ne aN © 
where the noise powers are assumed to be uncorrelated 
so they can be added directly. 
The output of a conventional pem system differs only 
with respect to the error noise power. The expression 
viven by (8) is still valid but with the simplification that 
the p; are equal since the a; are equal. The error noise 
power then can be written 


‘ a1 sel 
Ni=p 4 = z 
7=1 


P, (9) 


where the prime denotes the unweighted system. The 
output signal-to-noise ratio becomes 


Sy So 
= : 10 
(5 out N. te N, ( ) 
[The performance of conventional and weighted pcm 
systems are compared in Fig. 1 for various numbers of 
ligits per pulse-code group. 
Tue PRACTICAL WEIGHTED SYSTEM 


A practical system cannot achieve the weighting given 
vy (5) continuously since the pulse amplitude distribution 
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Fig. 3—Comparison of relative pulse amplitudes for practical and 
optimum seven-digit pem systems. 


then would become a function of the input signal power 
to the receiver, a quantity which fluctuates greatly in 
normal service. A practical solution is to select for the 
a; those values for a particular value of S and to accept 
the nonoptimum performance which occurs at other 
values of S. 

A logical choice for this design point is the value of 
input signal power at which the quantization and error 
noise powers are equal. This point is significant since 
for larger input signal powers the output noise power is 
determined principally by the quantization noise, while 
at smaller input signal powers the output noise power 
rises so rapidly that the system is virtually useless any- 
way. The relative pulse amplitudes corresponding to 
these design values have been computed using (5) and 
are shown in Fig. 2. These are actual weighting functions 
to be applied to the pulses at the transmitter. 

The effect of fixed weighting on the relative pulse 
amplitudes for a seven-digit system is illustrated in I’ig. 
3. The dotted lines show the weighting to be used at each 
input signal-to-noise ratio to achieve optimum perform- 
ance. The solid lines denote the fixed weighting which is 
adjusted to be optimum at the design input signal-to-noise 
ratio of approximately 11 db. It is seen that the optimum 
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weighting tends to level off at high input signal-to-noise 
ratios while the fixed weighting maintains the amplitude 
ratios established at the design input signal-to-noise 
ratio. The effect on the output signal-to-noise ratio is 
only apparent, however, since quantization noise soon 
predominates in either case. 

The performance of the practical system is computed 
in a fashion similar to the. preceding computations in 
that the output signal-to-noise ratio is written again as 


(8). ret 
NJ ote NG Ne 
except that the double prime on the error noise power 
denotes the use of the fixed pulse amplitude distribution. 
This noise power is given by the general form, (8), in 
which the p; are determined from the actual a; for each 
value of S as in Fig. 3 rather than from the optimum 
condition given by (5). The performance of a practical 
seven-digit weighted pem system is shown in Fig. 4. It is 
seen that the effect of fixing the relative pulse amplitudes 
to those determined at the design point is quite small 
compared with the general improvement of performance 
achieved by weighting. 


(11) 


INFORMATION RATE 


According to Shannon (Theorem 23), [7], information 
rate for any continuous source of average power Q and 
band W relative to rms measure of fidelity is bounded by 

Wlog 2 <R<Wihog ©, (12) 
where Q, is the entropy power of the source and N is the 
allowed mean-square error between the original and 
recovered messages. 

The entropy power of a source is defined as the average 
power of a normally distributed source having the same 
entropy as the original source. The pem systems discussed 
here are assumed to have uniform distributions of signal 
amplitudes. Let x denote a typical signal amplitude 
which lies in the interval (— a, a) with a probability 
density 


1 

Get) |r| < a 
p(x) = x 

ies pal a 


The source power Q is the mean-square value of x or 


(os) 2 
2 a 
() cA) Che = => 
Q ik. Bleed 3 
The entropy of the source is 


= — / p(x) log pla) dx = 


A normally distributed source has a probability density 
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where (, is its average power. Its entropy is 


py) = 


50 


45 
LIMITING CURVE 
WEIGHTED PCM 
40 


pr DESIG 
pointe 


ACTUAL 
WEIGHTED 


PCM 


25 7 DIGIT SYSTEMS 


(S/N), DB | 

7 

Fig. 4—Performance characteristics of a typical weighted pcm ~ 
system. 


so} | 
Pine -| p(y) log ply) dy = log V 2meQ,. (14) | 
Equating the entropies of (13) and (14) gives : 
6 
a. ==, (18) 


as the entropy power of the uniformly distributed pcm — 
source. 

The allowed mean-square error N in (12) is simply the — 
sum of the quantization and error noise powers while the — 
source band refers to the signal prior to coding as a pem — 
signal. With the help of (15), the bounded information 
rate of (12) can be written 


1 6/8 lie S 
on 8a 8 (5) So 2nW — Se 8 (3) 8) 


where R/2nW has the units of bits per symbol when the 
logarithm is taken to base 2 since a pcm channel transmits 
2nW symbols (or pulses) per second. 

By Shannon’s Theorem 17, the capacity of a channel of 
band W perturbed by white thermal noise of power N 
when the average transmitter power is limited to P is 
given by 


Je! 
C= W log (1 +7) 
where the capacity is the maximum information rate 


possible. For a channel of width nW cps the capacity 
becomes 
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“ig. 5—Information rates for typical weighted and unweighted 
pem systems. 
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bits per symbol. 
| The channel capacity of (17) and the bounded infor- 
mation rate of (16) are plotted in Fig. 5 for the seven- 
ligit system of Fig. 4. This plot not only demonstrates 
the essential shape of the curves of Fig. 4 by indicating 
un input signal-to-noise ratio below which the performance 
leteriorates rapidly, but it also shows how good pem is 
vhen compared with the performance of the “ultimate” 
system. Also, it shows that the process of weighting 
achieves a significant improvement in terms of the 
maximum improvement possible. According to Fig. 4, 
eighting moves the knee of the output signal-to-noise 
atio curve about 1.4 db to the left, an unimpressive 
mount in itself. On the other hand, Fig. 5 shows that the 
ee of the information-rate curve has moved 1.4 db 
ut of a maximum possible 8 db. Generally it is conceded 
hat significant approaches to the channel capacity cannot 
e made except by recourse to considerable circuitry 
| anes and message delay. 
It is instructive to compute the information rate on the 
asis of the per-symbol equivocation considering the 
ransmission channel alone. The source is a device generat- 
ng positive and negative pulses with equal likelihood 
‘iving a source entropy of one bit per symbol. In a con- 
rentional pem transmission all pulses have the same 
ay p of being in error so the equivocation is 


imply * 
Aa) = —[plogp + (1 — p) log (1 — pj]. 18) 


or weighted pem the pulses within the pulse groups have 
iffering error probabilities so the equivocation becomes 


10) = —> Dp. oer. + (Ap) og (1 pd]. (19) 


The information rates [H(z) — H,(x)] for the seven- 
igit systems considered are also shown in Fig. 5. First, 
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it is clear that there is no quantitative relationship 
between the rms fidelity and equivocation criterion 
rates in the region where errors are significant. Second, 
it is seen that weighting increases the information rate 
relative to an rms fidelity criterion but decreases it on an 
equivocation basis. This fact is not startling when it is 
recalled that the purpose of weighting is to decrease the 
net effect of errors rather than their total number, but it 
is disturbing when one realizes that exactly opposite 
effects can be predicted if care is not exercised in selecting 
the criterion on which a system is to be judged. 


CONCLUSION 


A well-designed pem communications system in- 
variably includes a comfortable safety margin, 7.e., an 
average input signal-to-noise ratio 20 to 30 db above the 
threshold, to protect the link from failure in all but the 
deepest fades. Under such conditions the 1.4 db or so by 
which weighting reduces the threshold may not be an 
important enough factor to justify its use if excessive 
circuitry complications result. It may be that a scheme 
in which, for example, the weighted pulses are transmitted 
in frequency-division-multiplex does not constitute such 
a complication, but the acceptance of a technique such 
as weighting must depend ultimately on the more ex- 
perienced considerations of the practicing systems engi- 
neers. Regardless of its practicability, however, the 
process itself is of interest theoretically because it improves 
the system performance without recourse to redundancy 
and processing-time lags for error correcting. 

The information rate study shows that the 1.4-db 
improvement mentioned above represents a significant 
approach to the channel capacity. Furthermore, it brings 
out. the importance of looking beyond the transmission 
channel itself when selecting the system worth criterion. 
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Radar Detection Probability with‘ Logarithmic Detectors’ 


BEN A. GREEN, JR.+ 


Summary—tIt is shown that use of a logarithmic (rather than 
square-law) detector in a search radar system inflicts a loss of 
sensitivity equivalent to a power loss of the order of one db under 
typical conditions. Curves of probability of detection vs relative 
range are given for various false alarm probabilities and various 
numbers of pulses integrated. The power loss (in db) is roughly 
proportional to the logarithm of the number of pulses integrated. 


INTRODUCTION 
LOGARITHMIC detector is a device which 
abstracts from an incoming high-frequency 


signal the logarithm of its envelope. Radar 
receivers employing a logarithmic detector are interesting 
to radar designers because they provide wide ‘‘dynamic 
range” and because they facilitate analog calculation of 
products, ratios, and powers. 

But one may wonder what effect this type of detector 
has on the “sensitivity” of the receiver; more precisely, 
what happens to the probability of detection for a fixed 
probability of false alarm? If only one pulse is considered 
at a time, then the answer is that the sensitivity is pre- 
cisely the same as that when any ordinary detector is 
used. But a difference appears when pulses are “inte- 
erated,” or summed, after detection in order to enhance 
the snr. The purpose of this work was to calculate the 
difference. 

This work is an extension of the work of Marcum,’ 
who calculated the probability of detection after inte- 
eration of pulses from a ‘“‘square-law”’ detector. 

A paper by Croney” gives the mean and standard 
deviation of the output of a logarithmic receiver when 
the input is thermal noise. The present work extends the 
calculation to higher moments of the distribution. 

Our approach is to approximate the distribution of the 
integrated pulses when they contain a signal by a normal 
distribution, whose parameters are calculated; when they 
do not contain a signal the distribution is calculated 
numerically or approximated by an Edgeworth series, 
depending on the number of pulses integrated. 


CALCULATIONS 


We write the probability density for the amplitude of 
the envelope of a high-frequency signal containing a sine 
wave of amplitude a and thermal noise of unit rms ampli- 
tude as follows: 


* Manuscript received by the PGIT, September 12, 1957. This 
work was performed by Bendix Radio Corp. under contract with 
Cornell Aeronautical Labs. 

+ Electro Metallurgical Co., Niagara Falls, N. Y. Formerly with 
Bendix Radio Corp., Towson, Md. 

1J. I. Marcum, “A Statistical Theory of Target Detection by 
Pulsed Radar,” U.S.A.F., Project RAND, RAND Corp., Santa 
Monica, Calif., Res. Memo. RM-754; 1947. 

2 J. Croney, “Clutter on radar displays,’’ Wireless Eng., vol. 33, 
pp. 83-96; April, 1956. | 

38. O. Rice, “Mathematical analysis of random noise,” Bell 
Sys. Tech. J., vol. 24, p. 101; January, 1945. 


me 2 
P(ala) = 2I,(ax) exp (-=+*), 


where J,(x) is the modified Bessel function of the first | 
kind of order zero. (We allow only positive x.) From this _ 
we may, when necessary, calculate Q(z|a), the correspond- 


ing probability density for the logarithm of this envelope. 
Further we may calculate Q,(a\a), that for the sum of k 
such pulses are samples of the envelope. 

The usual detection philosophy is to assign a certain 
small probability Py to the occurrence of a false alarm 
during one independent time interval. Then one agrees 
to announce the presence of a signal whenever the receiver 
output exceeds a certain threshold V, set to make the 
probability that noise alone will exceed it equal to Py. 
Thus, V is set from 


Py = ie Q.(x|0) da. 


Then the probability Pp that a sine wave of amplitude 


a will cause the alarm to be sounded is 


Py = iL Q,(a\a) dx. 


We now consider the first part of the problem: to 
calculate the threshold V. For pure noise, a = 0, and we 
have 


P(z|0) = 
If we call y = In a, then by a standard argument, 


Q(y|0) dy = P(a|0) dx. 


x exp (—2’/2). 


Thus, 
Q(y|0) = exp (2y — 1/2¢””) 


where y can be positive or negative. 
To calculate Q,o(x|0), the density for the sum of ten 


noise pulses, we resorted to numerical integration, a 


ten-fold convolution* of Q(a\0) with itself. This was 


carried out on the Bendix G-15 general purpose digital — 


computer by the Bendix Radio computing staff. 

To calculate Qjoo(z|0) numerically was not feasible. 
It was possible, however, to employ the Edgeworth 
series,’ an expansion of the density in terms of the normal 
distribution density and its derivatives. It was proved 
that the “cumulants’”® of Q(a|0) were given by’ 


, = (—2)- $n) -(m — I)! 


“H. Cramer, “Mathematical Methods of Statistics,’’ Princeton 
University Press, Princeton, N. J., p. 191; 1951. 

5 Tbid, p. 229. 

6 Ibid, p. 186. 

7 Derivation due to James Dokas, Bendix Radio Corp., Towson, 
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yhere ¢(n) is the Riemann zeta-function. By such means 
)1o0(x\0) was approximated sufficiently well to calculate 
he threshold for Py = 10°° and 10°’. These results were 
xtrapolated for Py = 10°", since the series begins to 
iverge, but the threshold is relatively insensitive to Py. 
Now for the second half of the problem: to calculate 
»p, the probability of detection for various values of Py 
nd the signal amplitude a. It may be shown that as a 
rows large, P(x|0) approaches a normal distribution 
ensity with mean, a. Further, the density Q(x|a) becomes 
ormal in the vicinity of « = In a. And, finally, since 
e sum of several identical variables is more nearly 
ormally distributed than any one, we feel safe in approxi- 
nating Q,(a\a) by a normal density. This approximation 
s best near x = In a; its failure becomes evident in the 
ails. In what follows we claim significant results only 
ver the range, 


0.01 < Pp <.0.99. 


“his is not a serious limitation. 
One must then calculate the mean m and standard 
eviation o of Q(z\a). It is simpler to calculate 


Me= / Gir ecria) de 


M, = in (In 2)’P(zxla) dz, 


| 
| 


‘ad then assert 
m= M,., 


_ Surprisingly enough, it is not difficult to get a series 
epresentation of M@, and M, in powers of a’. Briefly, 
me uses Rice’s expression® for the moments of P(a|a) 


and o = M, — Mi. 


} 
i 
| 


fn) = _ x"P(x\a) dx 


= 2"? .T(n/2 ae 1)-,F,(—n/2; ihe a /2). 


‘his is an entire function of n;° thus, we may differentiate 
if respect to n and set n equal to zero. This gives M,, 

may be seen by inspection. The second derivative 
aluated at n = 0 gives M,. The series for the hyper- 
sometric function was differentiated term by term with 
spect to 7, giving rise to a series for M, and M,. In fact 


M, = ina + 1/2 | aa 

aye t 
he integral is tabulated.’ M, could not be expressed in 
rms of tabulated functions. For large values of a, the 
ean approaches In a and the standard deviation ap- 
roaches a *. Fig. 1 shows the mean and standard deviation 
ad their asymptotes. 


18 Rice, op. czt., p. 101. ; 

°H. Bateman, ‘Higher Transcendental Functions,’ McGraw- 
ill Book Co., Inc., New York, N. Y., vol. 1, p. 260; 1953. Compiled 
Bateman Manuscript Project Staff. ; 
10}, Jahnke and F. Emde, ‘Tables of Functions,” Dover Publi- 
tions, New York, N. Y., p. 6; 1945. 
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Fig. 1—The mean, m, and standard deviation, oc, for the distribution 
of the output of a logarithmic detector (to base e) when the 
input is the sum of a sine wave of amplitude, a, and thermal 
noise of unit rms value. The asymptotic values, In a and 1/a, 
respectively, are also shown. 


This completes the work except for a minor compli- 
cation. We saw fit to account for the effect of the antenna 
pattern, which modulates the sequence of pulses as the 
antenna sweeps through the target azimuth. We assumed 
that the pattern was Gaussian in shape and that the k 
pulses were equally spaced between its “two-way half- 
voltage’ points. Thus, the mean and standard deviation 
of the sum of the k pulses are sums weighted by the 
Gaussian pattern. 

This makes it a little awkward to compare results for 
the logarithmic detector with those for the square-law 
detector, since Marcum assumed a “square” pattern, 
but it should suffice merely to adjust the value of a in 
Marcum’s results so that the average power received is 
equal in the two cases. 


Discussion 


The results are shown in Figs. 2 and 3. The abscissas 
are the relative range R/Ro, where Ry is that range for 
which the signal-to-noise power ratio is unity (with the 
antenna broadside to the target). Thus, assuming a 
fourth power range law, we have 


EL (3/2) = 0.7070)". 

Ro 

Fig. 2 shows the probability of detection when Py = 10°° 
for the two cases k = 10 and k = 100 pulses integrated. 
Fig. 3 shows the same when Py = 10 *° and includes 
Marcum’s results for the square-law detector as dashed 
lines. - 

We conclude that one suffers some loss of sensitivity 
by use of a logarithmic detector. The amount of the 
loss may be stated in terms of the equivalent loss of 
power; that is, the reduction in power of a conventional 
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Probability of Detection — per cent 


Relative Range — R/Ro 


Fig. 2—The probability of detection vs relative range after k pulses 
are integrated; false alarm probability, 10~. 


system which inflicts the same loss of sensitivity. For 10 
pulses integrated the loss is about 0.5 db; for 100 pulses 
integrated, about 1.0 db. We may also say, for 1 pulse 
the loss is zero. Thus, crudely, it appears that the loss 
(in db) is proportional to the logarithm of the number of 
pulses integrated. 

It is very likely that in most cases the advantages of 
a logarithmic detector will far outweigh the power loss. 


CORRECTION 


Saburo Muroga, author of “On the Capacity of A 
Noisy Continuous Channel,” which appeared on pages 
44-51 of the March, 1957 issue of these TRANSACTIONS, 
has requested the editors to make the following corrections 
to his paper, which were suggested by Prof. Y. Moriwaki 
of Tokyo University. 

On page 47, (39) should be 


C = —H(n) + zlog (er/y) 
and (41) should read 

y= HP + — (Ay). 
Eq. (42) should be 


AY 


Mare 


99 


Relative Range — R/Ro 


Fig. 3—The probability of detection vs relative range after k pulse 
are integrated, false alarm probability, 10~!°. The solid lines ar 
for the logarithmic detector; the dashed ones, for the square-law 
detector, (taken from Marcum,! and shifted to account fo 
Gaussian antenna pattern). 
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C = 2W[—H(n) + Flog, (/y)e] 
and (45) should be changed to 


ym oon cane 
C= W| low, : (1 Hee 


The term P in (46) and (47) should be replaced by P + 2N. 
Two lines above (48), the second equation should be 
y = —3(P + N). In the first paragraph on the right side 
of page 48, the footnote references to Muroga’s work 
should be 1 and 3. 
On page 51, two lines above (71), the term y’ should be 


_ Summary—Rice’s formula! for the ‘‘envelope” of a given signal 
very cumbersome; in any case where the signal is not a single 
ine wave, the analytical use and explicit calculation of the envelope 
practically prohibitive. A different formula for the envelope is 
iven herein which is much simpler and easier to handle analyti- 
lly. We show precisely that if a(¢) is the Hilbert transform of u(t), 
en Rice’s envelope of u(t) is the absolute value of the complex- 
alued function u(t) + i a(t). The function u + iw is called the pre- 
velope of u and is shown to be involved implicitly in some other 
sual engineering practices. 
The Hilbert transform w is then studied; it is shown that u has 
e€ same power spectrum as u and is uncorrelated with u at the 
ame time instant. Further, the autocorrelation of the pre-envelope 
if u is twice the pre-envelope of the autocorrelation of u. 
By using the pre-envelope, the envelope of the output of a linear 
iter is easily calculated, and this is used to compute the first 
irobability density for the envelope of the output of an arbitrary 
imear filter when the input is an arbitrary signal plus Gaussian 
‘oise. An application of pre-envelopes to the frequency modulation of 
m arbitrary waveform by another arbitrary waveform is also given. 
{ 
| 


Srecrion I 


E recall Rice’s formulation” of the envelope of 
a multichromatic® waveform w(t). He starts 
by writing u(t) in the form 


u(t) = >>, cos (at + ¢,) 
nd then selects a frequency g called the ‘midband 


requency.” Using this selected frequency, he writes 


SS C, COS [wn a gt = Pn AF qt] 


n 


QUAD = 


I, cos gt — I; sin gt 


pcre 
I, = de, cos [@ — Ot + vi] 


I, = Doe,sin [@, — dt+ el. 


“he expression 


RW) = Ue + Ty” 


termed by Rice the ‘envelope of u(t) referred to fre- 
uency gq.” 
This formulation has several defects. First, to obtain 
e envelope, one must expand the given multichromatic 
aveform in the form above. Second, one must select a 
midband frequency” gq; it is not immediately evident 
yhether or not a choice q’ # gq leads to the same R(t). 
eA the explicit calculation of R(f) is a formidable 
sk. 


* Manuscript received by the PGIT, May 5, 1957. This work was 
Une while the author was consultant to RCA, Los Angeles, Calif. 
+ University of Southern California, Los Angeles, Calif. 

18. O. Rice, Bell Sys. Tech. J., vol. 23, p. 1; 1944. 

2 [bid., p. 81. f : 

3 That is a waveform that can be written in the form )>-, Cy 


i (@nt + Gn), — 2 <t<4+o. 
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Envelopes and Pre-Envelopes of Real Waveforms’ 


J. DUGUNDJIt 


In this paper, we intend to give an equivalent, but 
more direct formulation of the concept “envelope.” This 
is done by introducing the idea of the pre-envelope of a 
given real waveform, which is a complex-valued function 
whose absolute value turns out to be exactly the envelope 
of the given waveform. Applications of the pre-envelope 
are given: 1) to show that A(t) depends only on the given 
waveform and not on the seemingly extraneous concept 
of “midband frequency,”* 2) to give direct methods for 
obtaining the envelope of the output of a linear filter 
when the input is an arbitrary waveform, 3) to calculate 
explicitly the first probability density for the envelope 
of the output of an arbitrary linear filter when the input 
is an arbitrary signal plus Gaussian noise, and 4) to give 
a definition of frequency modulation when both the 
waveform u(t) and the modulating function m/(t) are 
arbitrary. 

The paper is divided into six parts. Section II contains 
the essential facts about Hilbert transforms that are .- 
used; proofs will be found in Titchmarsh.’ In Section III 
the definitions of pre-envelope and envelope are given, 
and the equivalence of our envelope with that of Rice is 
established. Section IV contains further properties of 
pre-envelopes and, in particular, shows how one is 
naturally led to their consideration. In Sections V-VII 
the applications cited above are given. 

We remark that Hilbert transforms have been used in 
electrical engineering before, notably by Gabor,° Wood- 
ward,’ and Ville;> the concept “pre-envelope’”’ is in fact 
identical with the “signal analytique” of Ville, but our 
use and purpose differ from his. 


Srcrion II 


Given a real-valued function u(t) on — © <t< + o, 
its Hilbert transform @(t) is defined by 


, i eee 
i a 


where the principal value of the integral is always used. 
All functions considered here will be assumed to have 
Hilbert transforms. One readily verifies 
A. The Hilbert transform of cos(wt + g) is sin(wt + ¢). 
It is proved in Titchmarsh’ that, under suitable con- 
ditions, 4 = —u. 


4 This independence of ‘‘midband frequency’’ can of course also 
be seen directly from Rice’s definition without using pre-envelopes. 

5H. C. Titchmarsh, ‘Introduction to the Theory of Fourier 
Integrals,’ Oxford University Press, New York, N. Y.; 1937. 

6 1). Gabor, J.JEHE, pt. 3, vol. 93, p. 429; 1946. 

7™P. M. Woodward, ‘Probability and Information Theory,” 
McGraw-Hill Book Co., Inc., New York, N. Y.; 1953. 

8 J. A. Ville, “Cables et Transmissions,” (2nd ed.) no. 1, p. 16; 
1948. 
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B. The Hilbert transform of a Hilbert transform is 
the negative of the original function. 

C. If U(f) = Jf2. u(t) exp (— 2mift)dt is the Fourier 
transform of u(t), then the Fourier transform L/(f) of 
a(t) is 

[2 0 
Ey ==> 0 (prowl) a 
Ree i 20: 


D. The convolution 


fo) 


w() =) ul) = [ult — 8 ak 


has Hilbert transform 


w(t) = v(t) * ad). 


Srecrion III 


Definition: Let u(t) be a real waveform. The pre- 
envelope z(t) of u(t) is the complex-valued function 
2(t) = u(t) + 7a(t). The envelope of u(t) is the absolute 
value | 2(t) | of its pre-envelope. 


1) This definition of envelope gives the same result 
as that of Rice, whenever Rice’s definition is applicable. 

In fact, with the notations of Section I, we start by 
writing w(¢) in the form 


TG) ee SS Cn COS (w,t + ¢,). 


Forming the pre-envelope a(t) of u(t) by using A., one gets 
z(t) = >> ¢, cos @t + ¢,) +7 doc, sin (@t + ¢,). 
Referring to the frequency qg and noting that 
> sin [@, — @gt+¢, + at] = I. cos gt + I, sin qt, 


one finds 
z(t) = [J. cos gt — I, sin qt] + 27, cos gt + I, sim gt] 
= [I, + u,] exp [zgt] 
which shows that 
Ie)| = RO) 
and establishes the result. 


Since z(t) depends only on w(t), one obtains the following 
corollary. 


Corollary. Rice’s envelope is completely independent of 
the “midband frequency” q that is selected. 


SrcTion LV 


A motivation for using pre-envelopes is given here. 
Further, the power spectrum of @ and the cross correlation 
of u, @ are calculated; these will be needed in the sequel. 

The motivation for Hilbert transforms comes from 2) 
and 3) as follows: 


Mare 
2) The Fourier transform of zg = u + vis 
2U(f) fe=50 
Zf) ==) ea meee 
0 (fp D 


where U(f) is the Fourier transform of w(t). 
Since Z(f) = U(f) + iL(f), where L is the Fourier 

transform of 7, the result is immediate from Section II-C. 
The important thing is that the inverse of 2) is also 

true. 


3) Let 2(¢) be any (necessarily complex-valued) function 
with Fourier transform Z(f) vanishing for all f < 0 
Then z is the pre-envelope of its real part. That is, i 


u = real part of z, then z = wu + 24. 
To see this, define 
2Z(f) how. 
UG) = 0 Jom 0 
Lid Gos Mt <7) 


where the asterisk denotes (as it will throughout this paper) 
“complex conjugate.” Setting 


ult) = [ UC) exp [+2nift| af, 


one easily shows that u(t) = u*(t), so that wu is real- 
valued. Form the pre-envelope » = u + 7 of u. By 2), 
its Fourier transform is precisely Z(f) except possibly at 
f = 0, and this implies at once that w = 2. 


Now, it is standard engineering practice in considering 
frequency spectra to double the positive frequencies and 
cut the negative ones on the grounds that the positiv 
frequencies are the physically meaningful ones and th 
(mathematical) negative frequencies merely reflect the 
positive ones in complex conjugate form. From 2) and 3), 
it is evident that the mathematical counterpart of this 
physical practice is to form the pre-envelope of the 
waveform considered. The fact of 1) is a further reinforce- 
ment of the utility of the concept of “pre-envelope.” 

For any two waveforms wu, v, whether real or complex- 
valued, their cross correlation R,, is defined by 


_ Out + 8 a 


and their cross-power spectrum W,,, is the Fourier trans- 
TORMPOLuee 


Wilf) = / RG) exp [a oende 
It is well known’ that 


E. R.,(0) aa hee —t). 


* James, Nichols, and Phillips, “Theory of Servomechanisms,” 
M.LT. Rad. Lab. Ser., McGraw-Hill Book Co., Inc., New York, 
N. Y., vol. 25; 1950. See p. 279. 
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1e autocorrelation R,,, of wis written simply R,,. 


4) The cross correlation R,,, is precisely Re the Hilbert 
ansform of R,,. 
Tn fact, rewriting ,(¢) in the form 


* ult 
ad) = 4 i Has ©) dx, 
Jomo - —2 


1¢ has, since w is real-valued, 


1 ab uO - x + D4 


aaa — 


Rua) = = lim 55 


ssuming interchangeability of the order of integration 
id limits gives 


Raat) 2 if 4 dx 


oe oT te ulgyu(a + t + &) ae} 


To0 


dx = R(t) 


SN eR (ad) 
ae = 


hich establishes the result. 


Corollary. Ry2(—t) = —R,,(t). In particular, u and &% 
e completely uncorrelated at the same time instant, 
ry R,a(0) = 

For, by using E. and observing that R,, is real-valued, 


he has 
S. if R(t — 2) ae 
Tele a a 


hd changing the variable of integration to § = — 2 
ives the desired statement. 


Rui(—#) = =[ Fle —D ay 


Corollary. The cross-power spectrum of wu and % is 


ae, Muy een U 
W.alf) = 0 f=0 
+1W.(f) f <0. 


is is immediate from C. and 4). 
We now turn to the study of R, and W,. 


5) wand @ have precisely the same autocorrelation and 
pwer spectrum. 
From 4) one has 
and its Hilbert transform a= 
rain that 


R(t) = Rut). Now, starting from 
— u, one finds from 4) 


Red) = Re a(t) = —R,(0). 


; ing E. and the first corollary above shows 
Rit) = —Ru(—t) = BAL) 


ich is what was asserted. 
For the pre-envelope itself, one gets immediately from 
and 5) the following: 


6) Let 2(t) = u(t) + 7a). Then 


R.At) = 2K) + ih] 
[re 0 

W.f) =12W.f)  f=0 
| 0 jr" 


Srecrion V 
Application to filters is now given. 


7) Let u(t) be a real input to a linear filter having a 
real impulsive response function r(t). Then the pre- 
envelope of the output is obtained by taking the pre-en- 
velope of wu as input. 


In fact, with the pre-envelope z = u + it as input, 


one obtains for output the convolution 
w=r*z=r*utiur tt. 


But w = r*u is the output when wu alone is input, and 
by D. one has r+@ = ty. Thus, w = uy + ito, proving 
the assertion. 


Corollary. In terms of the frequency response function 
Y(f) of the filter, the pre-envelope of the output when w is 
taken for input is 


2a ” Y(U() exp (Qnift) df 


where U(f) is the Fourier transform of u(t). 

For, by using 7), one need only compute the output 
when the pre-envelope z = wu + 7é is input. Since the 
Fourier transform of a filter output equals the Fourier 
transform of the filter input times frequency response 
function, the output in the case to hand has Fourier 
transform Y(f)Z(f). By 2), 


P VQCG rec 
YNZ) =) YYUG~ f=0 
| sparen 


Defining Y(0)U(O) = 0, which does not affect the inverse 
Fourier transform of Y(f)Z(f), one has for output the 
desired formula. 


Using a shghtly different viewpoint, this corollary 
says that the pre-envelope of the output is obtained by 
taking wu alone as input and redefining the filter frequency 
response by doubling the positive frequencies and killing 
the negative ones. As another simple application. 


8) Let u(t) be a waveform having frequency spectrum 
in the bands fy — 3W <|f| < fo + 4W. Then the square 
of the envelope is frequency limited to | f | SNe 

Indeed, if 2 = u + 7é is the pre-envelope of u(t), we 
are interested in 


KGS Me le(t)|” exp [—2z7ft] dt 
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Now, N and W are independent, so that their joint probabilit 


= i 2(t)z*(t) exp [—2mift] dt = Z(f) * Z*(—f) 


fot hw 
fh -1w 


and this is readily seen, by 2), to vanish outside 


Z(E)Z*(E — f) dé 


{|< W. 


We remark that the hypotheses of 8) do not allow one to 
draw the conclusion that the envelope itself is band- 
limited; indeed, there seems to be no physical reason 
that it should be so. 


Srcrion VI 


The first probability density for the envelope of the 
output of a linear filter, when the input is arbitrary 
signal plus Gaussian noise with zero mean, will be 
computed. 

It is well known”® that if Gaussian noise having zero 
mean and power spectrum W(f) is passed through a 
linear filter having frequency response Y(f), then the 
output 1) is Gaussian with zero mean and 2) has power 
spectrum | VG) le W(f). From this 2) we find at once 
that the variance of the output distribution, which is the 
value of the autocorrelation at ¢ = 0, is 


foo) 


fe i _LY@OP WO) af. 


The probability density of the output N is therefore 


Lats exp [ | 
o WV Qa 20 | 

Let {n(t)} be sample functions for the noise, and 
consider the two-dimensional stochastic process Q whose 
samples are the ordered pairs [n({), “(f)], the second 
term always being the Hilbert transform of the first. 
Observe that if 1 (f) is the output of the filter when 
n(t) is input, then by 7) the output when “”(t) is applied 
is the Hilbert transform /,(t) of no(t). The two-dimensional 
distribution of the output (NV, V) when the Q process is 
put through the filter is now to be determined. 

We have seen that N is Gaussian with zero mean and 
variance o. Now Wis also Gaussian with zero mean and 
variance co’. That NV is Gaussian with zero mean is seen 
by using for the {n(t)} the same functions as does Rice” 
and applying A. to each. That the variance of NV is o° 
results from 5), since this tells us that the functions 
{n(t)} have the same power spectrum as the {n(t)}. 

Putting the sample [n(¢), %(¢)] through the filter gives, 
as has been remarked, the output [7(t), %.(f)], the second 
variable being the Hilbert transform of the first. According 
to the corollary in 4) n(t) and %,(t) are always uncorrel- 
ated at the same instant of time. It follows at once that 


10 Tbid., p. 288. 
1 Rice, op. cit., p. 48. 


- where, according to the corollary of 7) one has uw) + 


density is given by 


N? lis L) 


a it : = 
pV, 9) = 54x exp| ) 


With the preliminaries aside, we prove the following: 


9) Let signal w(t) plus Gaussian noise with zero mean and 
power spectrum W(f) be put through a filter having 
frequency response function Y(f). Then the envelope R& 
of the output has first probability density 


1) = ¥oxp[ B+ HOE, [Bata 


where J, = Bessel function of zero order and purely 
imaginary argument, 


iS) 


CGC = 


[1 7@ Pwo ar, 


|e) | = 2 | ie Y(U(f) exp [2rafi] df | 


= envelope of output if signal alone is in= 
put. 

In fact, let wo be the output when w alone is input, and 
nm be the output when the sample function n(é) is input; 
then, by 7), the pre-envelope of the output of signal 
plus noise has sample functions (wo + mo) + 7(to + to), 


tty) = 2. The envelope of the output is therefore 


R = [(tto + No)” + (Go + Haye las 


In view of the previous discussion, the problem is reduced 
to calculating the probability density of 


R= [(%o + N)’ Seen Se NY 


where N, WN are distributed as in (1). 
The substitution 


R cos 6 = u+ N 
Rsnd=t%, +N 


I 


gives 
N? + N? = R? + us + tg — 2Rfwo cos 6 + My sin 6] 


and the method now used is the same as that of Rice.’? 
Observing that 


w+ @ = 2) 


R pels oe | 2 |? 
Onae ak L 20” 


“exp i (uy cos 6 + %& sin 0 | dR dé. 


one gets from (1) 


pk, 0) dk dé = 


2 Tbid., p. 106. 
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ne now integrates out the @ by setting 


Ug =p. COS) 


A 


yo =p sin p 


yhere p= | 2 |, verifying that uw cos 6 + uw sin 6 = 
cos (@ — yu), and then noting that, 

/ e*°°"" dé = QnI,(a). 

0 


The result is established. 


Section VII 


Application to frequency modulation will now be given. 
Definition: Let u(t) be a given real waveform. For any 
real waveform m/(t), we will say that the function 


u(t) = u(t) cos m(t) — a(t) sin m(t) 
5 u(t) frequency modulated by m(). 


| Before giving a justification of this definition, ob- 
i that if u(t) = cos 2rft and m(t) = m sin 2rat, 
then from A. it follows at once that our definition reduces 


wi) = 


© the usual definition of frequency modulation: 
jos (2rft + m sin 2rat). 

_ We also notice that if z(t) is the pre-envelope of w(t), 
ind if ® denotes “real part of,” then 


u(t) = R{2(t) exp (im(Z)}. 


However, we must remark that the complex wave z(t) = 
xp [7m(t)] is not the pre-envelope of u(t). 

One is led to this definition in the following natural 
vay. We have 


u(t) = R{| 2(d) | exp [¢¥(4)]} = | 2(#) | cos yd) 


here 


a(t) 
u(t) 


ind | 2(t) is the envelope of JOE Thus, u(t) has been 
yritten as a cosine wave with a “slowly” varying ampli- 
ude and an instantaneous pinecuency: frequency modu- 
ution in the usual sense’ says that the frequency 


hodulated wave is 


V@). = are tan... 


13 Note the meaning we attach to “frequency modulation;”’ this 
3 quite often called “phase: modulation.” In all events we can pass 
rom ‘“‘phase’’ to “frequency”’ modulation easily by using derivatives. 


W(t) + m(t)] 

RI] 2(t) | exp LOO) + m(O)]} 
{| 2(d) | exp [¢¥(d)]-exp [em(d)]} 
R{z(t) exp [em(t)]}. 


This physical reasoning, together with the fact that it 
coincides with the usual definition whenever both are 
applicable and because this proposed definition applies 
in cases where the usual one does not, leads us to propose 
our definition as a generalization of the usual notion of 
frequency modulation. 

As a simple application, we calculate the spectrum of 
an arbitrary real waveform u(t) frequency modulated by 
b sin 2rat. Let z(t) be the pre-envelope of w(t). Observe 
first that if I'(f) is the Fourier transform of the complex 
waveform y(t) = z(t) exp [ia sin 27at] then the spectrum 
of its real part, which is what we are seeking, is 3[['(f) + 
I'*(— f)]; thus it suffices to compute I'(f). Recalling that 


| e(t) | cos [ 


u(t) 


I 


I 


SD J,(b) exp (12rnat) 


n=—@ 


exp [7b sin 2rat] 


where J,, n > 0 is the ordinary Bessel function of order 
nand J_, (x) = (— 1)” J,(a), we have 


a/u)) — os J,(b)2(t) exp [72rnat]. 


If Z(f) is the Fourier transform of z(t) and 6(f) is the Dirac 
delta function, the convolution theorem gives 


x TA(d)Z 


To express this directly in terms of U(f), the Fourier 
transform of the given u(t) according to 2) is: 


Z(f) = (1 + sgn f)U(/) 


DD RAO 1): 


n=—0 


rf) = (f) * &(f — na) = 


where 
sist (pew, 
sen f=) Of 
seal) iO), 


and we find 


ri) = DL FO)[1 + san (f = na) UU = ned. 


As remarked, 4[T'(f) + I'*(— f)] is the required spectrum. 
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Correspondence 


On Weighted PCM and 
Mean-Square Deviation 


In an interesting paper Bedrosian has 
introduced the concept of weighted pulse- 
code modulation, wpem.! This differs from 
normal pem in that the amplitude of the 
transmitted pulses representing the binary 
digits in a pulse-code group are made to 
depend on the size of the group and on the 
power of two represented by the individual 
pulses. In general, the higher the power of 
two represented by the pulse, the larger the 
amplitude of the pulse. A comparison be- 
tween pem and wpem system performances, 
indicating the advantages of wepm, can be 
found in Bedrosian’s paper. 

1) The analysis of a wpem system in- 
volves the determination of the optimal 
power to be allocated to the transmission 
of each binary digit, under the assumption 
of a fixed power per pulse-code group, to 
minimize the mean-square deviation of the 
reconstructed signal at the receiver from 
the signal sample value presented by the 
information source. If there are N digits 
per pulse-code group, then use of the La- 
grange multiplier method involves the solu- 
tion of a system of N + 1 simultaneous 
transcendental equations. Bedrosian  ac- 
complishes this under the assumption that 
the average signal power available is suf- 
ficiently great to make the occurrence of 
more than one error per pulse-code group 
almost zero. This assumption leads to a 
system of V + 1 independent equations for 
the power allocations and the Lagrange 
multiplier. 

In addition, he assumes that the proba- 
bility of a digit’s being received incorrectly 
as a function of signal strength, and the 
derivative of this function, are both simply 
expressible in terms of the error function. 
This may lead to difficulties in locating the 
extrema since the smoothed Curve I in 
Fig. 1 may be a reasonable approximation 
to Curve II,-without the slope of I at a 
point necessarily being a reasonable ap- 
proximation to the slope of II at the same 
point. 

2) We wish to present an alternative 
approach to the analysis, based on the 
functional equation technique of dynamic 
programming,? in which no assumption of 
smallness is necessary concerning the prob- 
ability of error; this is significant for a 
wpem system may function satisfactorily 
with signal power so low that several errors 
per pulse group occur, the errors being 
localized to the coefficients of the lower 
powers of two, which is manifestly not 
possible with ordinary pem. In addition, 
minima are determined without the use of 
partial derivatives. This permits the proba- 
bility of error to be given as a function of 
signal strength either graphically or in 
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form. Lastly, 
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Fig, 1. 


tabular form, based upon experimental 
results, and not necessarily in analytic 
the quantization noise is 
handled in a particularly simple and 
straightforward fashion. 

3) Consider that each pulse-code group 
consists of N binary digits and that the 
power P is available for sending the entire 
group. The probability of the correct re- 
ception of a digit if the transmitted power 
is p is defined to be g(p). Assume that trans- 
mission and reception of the individual 
digits are independent. We wish to deter- 
mine the power p; to allocate to the trans- 
mission of the coefficient of the ith power 
of two which results in minimizing the 
mean-square deviation of the received 
signal from the signal sample value pro- 
vided by the source of information, where 


Si (1) 


i=0 
0 < Pi 


1S 1, 20> 22 Ne 2 be (2) 
Assume that the transmitter will transmit 
a “one” as the coefficient of 2V-1 if a > 
2N-1, where a is the signal sample value, 
and a “zero” if 0 < @ < 2-1; call this 
coefficient cy_; . It transmits a “one’’ as 
the coefficient of 2-2 if a — cy_, 2N-1 > 
2N-2, a “‘zero”’ otherwise, and so on. 

The reconstructed received signal will be 


Sy_-1 = adler 
+ 2 yo +--+ + Qa 
= Den + Sy-2, (3) 


where az; is a random variable assuming 
only the values zero or one. Since the 
probability of a correct transmission of the 
kth digit with power px is g(px), 


prob {a = cz} "= o@ins (4) 


The mean-square deviation of the received 
signal from the signal sample value a is 


E{(a — sya) } = BV @S 2 aye) | 
— Qla — EB {2 “ay_1} 1 {sy_o} 
4- E{sy-o} . (5) 


By completing the square, this becomes 


E{(a — sy-1)?} = E{(@ — 2°" ay1)J 
(e- 2 o 
+ Ei(a—# 


= Sy-2) } 


{ Aa a \ 


= var {a — 2” *2y_1} 


{ is\e—ah 


+ E{(a — B{2 
= Syaa) yy: (6) 


To transform the minimization problem 
into one involving functional equations, we 
introduce the function 
fy(a,P) = the expected value of the squar 
deviation of the received signal 
from the source signal @ using an — 
optimal allocation of the power 
P to the individual digits, eac 
pulse-code group consisting of 
digits. 

By definition, then, 


min E{(a — sya =fra,P), @ 


{pi} 


Un-1 } 


where the minimization is over the set of | 
p; defined by 


O <p, N04 et ee q 
poe a (8) 


If we wish to include the effect of a limita- 
tion on peak power Ppx, we may impose, in 
addition, the condition that 0 < p; < Pox. 
This would be difficult to incorporate in the 
Lagrange multiplier scheme. 

If we now use the principle of optimality? 
and (6), we obtain 


fy(a, jz) = 
min {var {a — 2° Sa, 
OSD yee 
ae ee (a = Bi 2es Ne ae fis ine Py-1)}, 


NS 2 oy eee (9) 


If the pulse-code group consists of just one 
binary digit, all available power is used and 
we obtain 


h(a, P) 
g(P)a* + (1 — g(P))A — a)’, 
~~ Ol Geaale (10) 
g(P)(L — a) +0 = gP)a7; 
Porter } 


’ \ 
Thus the multidimensional optimization — 
problem is converted into a sequence of | 
one-dimensional optimization problems. 


‘ 
: 


4) Using (9) and (10) we are able to 
pute recursively, using a high-speed 
ital computer, the sequence of functions 
two variables {fi(a,P)} and the power 
ocations pz as functions of k, a, and P. 
is computation is straightforward on a 
ital computer and is simplified by the 
t that if fy(a,P) is to be determined for 
nd P in a certain domain 0 < a < a, 
< P < Po, then all the fi(a,P) fork < N 
ed only be determined in subdomains, as 
;seen upon referring to (9). 

5) Note that the transmitter is not pro- 
ed with a feedback link which gives it 
iformation concerning whether or not the 
evious signal was correctly received, so 
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Continued expansion in our field of 
et was demonstrated again by the 
llowing recent announcement. 

An International Association for Cyber- 
ptics has been formed, with the Board of 
ldministration including members from 
elgium, France, the United Kingdom, and 
ue U.S. A. 

_A permanent secretariat has been set up 
ith offices in Namur, Belgium, and the 
rst International Congress on Cybernetics 
s held in Namur from June 26-29, 1956. 
ans for the second congress are being 
ade now for September, 1958. 
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that the exact state of the system is not 
known at the transmitter at each stage. 
Nonetheless, the transmitter is able to 
transmit optimally with regard to the least 
mean square deviation criterion by proceed- 
ing as if the signal a had been changed 
purely deterministically to a —H{2N-1xy_1}. 

This observation should find application 
in other situations involving least mean- 
square-error criteria. It means that in some 
situations an equivalent deterministic proc- 
ess can be associated with a _ stochastic 
process. 

6) The problems involved in optimizing 
the performance of null-zone reception sys- 
tems and null-zone reception systems pro- 
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vided with feedback channels? can be 

treated similarly, by use of the functional 

equation technique. It would be of interest 

to evaluate the performance of communi- 

cation systems employing combinations of 

wpem, null-zero reception, and feedback 
channels. 
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